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ABSTRACT 


TITLE  OF  THESIS:  The  Breakdown  of  the  PC  Paradigm:  A  Critical  Look  At 

Information  Display  Technology  As  An  Analysis  Inhibitor 

STUDENT:  Steven  F.  Rue,  GySgt  USMC 

CLASS  NO.  PGIP2003  DATE:  August  2003 

THESIS  COMMITTEE  CHAIR:  Mr.  Francis  J.  Hughes 
SECOND  COMMITTEE  MEMBER:  Dr.  John  H.  Spurlin 

The  Department  of  Defense  (DoD)  has  come  to  rely  on  Commercial-Off-The- 
Shelf  (COTS)  acquisition  as  the  primary  means  to  introduce  and  upgrade  computer 
technologies  throughout  the  military.  In  adopting  this  approach,  the  personal  computer 
(PC),  based  on  a  commercial,  business-model  design,  has  become  a  de-facto  standard. 

Within  the  thesis,  this  common,  and  widely- accepted  desktop  computer 
environment  is  defined  as  the  PC  Paradigm  and  encompasses  the  typical  office-suite 
software,  the  information  display,  and  the  use  of  the  desktop  metaphor  as  the  user- 
interface.  The  author  contends  that  this  near-exclusive  reliance  on  COTS  technologies  is 
based  on  market-driven,  commercial  product  solutions  rather  than  a  “domain  specific” 
system  design  that  is  optimized  to  support  intelligence  analysis. 

The  thesis  addresses  the  following  research  question:  Could  the  current  personal 
computer  environment  (characterized  as  the  PC  Paradigm)  function  as  an  analysis 
inhibitor,  rather  than  as  a  tool  to  facilitate  analysis? 

A  grounded  theory  approach  was  utilized  in  structuring  the  study.  The  research 
draws  upon  an  exhaustive  survey  of  cross-disciplinary  literature  encompassing  applicable 
research  in  the  fields  of  cognitive  science,  graphic  design,  visualization  display 


technologies,  and  human-computer  interface  design,  and  seeks  linkages  and  points  of 
intersection  between  the  related  fields  relevant  to  the  research  question. 

Key  findings  suggested  that  a  “paradigm  conflict”  exists  between  the  needs  of 
intelligence  analysis  and  the  commercial,  office-based  personal  computer  employing  the 
desktop  metaphor  as  a  user  interface.  Additionally,  COTS  technology  acquisition, 
and  its  associated  commercial  standards,  have  appeared  to  supplant  a  design  process 
optimized  for  the  needs  of  the  intelligence  analyst.  Such  an  approach  disregards  domain 
specificity  in  favor  of  a  generalized,  “one-size-fits-all”  product  solution. 

NATO  perspectives  and  design  initiatives  are  reviewed  and  contrasted  with 
current  Department  of  Defense  guidance.  Key  DoD  technical  architecture  documents 
reveal  a  conspicuous  lack  of  design  guidance  regarding  human-computer  interface  issues. 

The  thesis  concludes  that  the  PC  Paradigm  environment  does  contribute  to 
inhibited  analysis.  The  current  paradigm  is  not  derived  from  cognitive  ergonomic 
requirements  unique  to  intelligence  analysis,  nor  does  it  support  the  analytic 
methodologies  widely  regarded  as  “best  practices”  within  the  field.  The  author  contends 
that  domain  relevance  is  lacking  in  business-model  tools  when  applied  to  the  problems  of 
intelligence  analysis. 

The  thesis  provides  a  unique  convergence  of  concepts  and  design  considerations 
from  disciplines  outside  the  normal  scope  of  the  intelligence  community,  and  seeks  to 
provide  a  synthesis  of  multidisciplinary  research. 

Configuration  management  for  cognitive  systems  is  advocated,  as  well  as  the 
further  exploration  of  Cognitive  Task  Analysis  (CTA)  methodologies  as  tools  for 
assessing  both  proposed  and  existing  analyst  support  systems. 
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CHAPTER  1 


PROBLEMATIC  PARADIGMS:  A  CRITICAL  LOOK  AT  CURRENT 
INFORMATION  DISPLAYS  AS  TOOLS  FOR  INTELLIGENCE  ANALYSIS 


COMMERCIAL  OFF-THE-SHELF  SOLUTIONS:  FAST  AND  FLAWED 


Somewhere  in  the  midst  of  a  phenomenon  that  has  been  eharacterized  as  “the 

•j 

ruthless  pursuit  of  COTS,”  intelligence  analysis  became  imperiled.  Within  the 
Department  of  Defense  (DoD),  commercial-off-the-shelf  (or  COTS)  products  have  been 
increasingly  seen  as  the  “fast-track”  to  inject  technological  solutions  into  a  less-than- 
nimble  procurement  system.  And  while  research  and  development  costs  and  long  lead 
times  have  indeed  been  slashed,  the  wholesale  adoption  of  products  created  for  the 
commercial  market  has  eroded  the  military’s  ability  to  influence  and  manage  design. 

Presently,  both  DoD  and  the  private  sector  operate  under  what  the  author  terms 
the  PC  Paradigm^,  an  environment  driven  by  mass-market  imperatives  that  are  based 
upon  a  business-model  design.  This  thesis  argues  that  the  widespread  acceptance  of  the 
commonly  available  desktop  PC  as  an  analyst  tool  is  problematic  and  suggests  it  can 
function  as  an  analysis  inhibitor,  masking  alternative  representations  of  the  data  under 
consideration.  Such  implications  are  non-trivial.  If  “analysis”  is  regarded  as  a  subset 

*  North  Atlantic  Treaty  Organization,  Commercial  Ojf-the-Shelf  Products  in  Defence 
Applications  “The  Ruthless  Pursuit  of  COTS,”  RTO-MP-048  (Neuilly-Sur-Seine  Cedex,  France;  Research 
and  Technology  Organization,  December  2000),  vii.  Cited  hereafter  as  NATO,  COTS  Study. 

^  The  author’s  use  of  the  term,  PC-Paradigm,  denotes  a  “Personal  Computer  Paradigm”  that  is 
used  to  characterize  the  conventional,  commercially  available  desktop  or  laptop  personal  computer 
configuration  to  include  its  Windows-based  desktop  metaphor.  The  concept  is  further  elaborated  upon 
within  the  thesis. 
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within  the  domain  of  intelligence,  such  an  effect  could  potentially  create  conditions  of 

analytical  domain  restriction,  inadvertently  guiding  and  limiting  the  analyst’s  thought 

processes  and  solution-sets  to  those  facilitated  by  the  tool. 

The  author  contends  that  the  fundamental  rift  between  the  use  of  the  personal 

computer  (PC)  as  an  “analytic  tool”  and  its  suitability  to  perform  in  that  role  is  largely 

based  on  failure  to  employ  widely-accepted  elements  of  design;  particularly  cognitive 

factors  and  human -computer  interface  (HCI)  principles.  By  surrendering  to  COTS 

acquisition,  DoD  has  in  effect  endorsed  a  design-by-proxy  philosophy  that  has  transferred 

responsibility  and  thus  the  design  initiative  to  private  industry,  contractors,  and  vendors. 

Current  practice  closely  resembles  what  has  been  referred  to  as  the  anticipation 

and  control  dilemma,  as  articulated  in  a  recent  study  conducted  by  the  Advanced  Science 

and  Technology  Policy  Planning  Network  for  the  European  Union: 

At  an  early  stage  of  technology  development,  the  nature  of  the  technology  (and 
the  articulation  of  interests)  are  still  malleable  -  but  it  is  unclear  what  the  effects 
and  impacts  will  be.  By  the  time  these  become  clear,  the  technology  is 
entrenched  and  vested  interests  make  it  difficult  to  change  the  technology. 

Such  criticism  is  typically  addressed  by  incremental  design  changes,  hardware 
upgrades,  and  the  ubiquitous  new  software  release.  In  actual  practice  such  solutions 
result  in  “feature  creep”  and  what  has  been  characterized  as  “development-by¬ 
accumulation. Incrementalism,  however,  only  serves  to  ornament  a  fundamentally- 
flawed  design. 


^  Stefan  Kuhlman  and  others.  Improving  Distributed  Intelligence  in  Complex  Innovation  Systems: 
Final  Report  of  the  Advanced  Science  and  Technology  Policy  Planning  Network  (ASTPP),  Monograph, 
Fraunhofer  Institute  (Karlsruhe,  Germany:  Institute  Systems  and  Innovation  Research,  June  1999),  42. 

*  Thomas  S.  Kuhn,  The  Structure  of  Scientific  Revolutions,  3"*  ed.,  (Chicago:  The  University  of 
Chicago  Press,  1996),  2. 
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In  keeping  with  Kuhn’s  classic  perspectives  on  the  concept  of  paradigm^  it  is 
instructive  to  note,  that 

a  new  theory,  however  special  its  range  of  application,  is  seldom  or  never  just  an 
increment  to  what  is  already  known.  Its  assimilation  requires  the  reconstruction 
of  prior  theory  and  the  re-evaluation  of  prior  fact,  an  intrinsically  revolutionary 
process.^ 

Such  revolutionary  measures  are  called  for  in  this  study.  If  the  intelligence 
community  is  to  achieve  analytical  dominance  in  the  face  of  ever-more-sophisticated 
adversaries,  it  will  be  unlikely  to  do  so  equipped  with  “over-the-counter”  COTS 
technologies.  One  path  towards  this  goal  may  be  to  seek  out  productive  ways  in  which 
to  capitalize  on  the  best  practices  of  multiple  domains.  Fusing  the  IC’s  understanding  of 
the  intelligence  domain  with  design  principles  derived  from  promising  research  in 
cognitive  science,  human-computer  interface  design,  and  visualization,  could  create  a 
revolutionary  synthesis.  The  goal  is  to  regain  design  initiative. 

It  is  hoped  that  the  evidence  presented  in  this  study  will  contribute  to  the 
formation  of  new  ideas,  provide  new  linkages  among  existing  data,  and  illuminate  the 
shortcomings  of  the  current  PC  Paradigm-,  in  short,  to  “to  urge  a  change  in  the 
perception  and  evaluation  of  familiar  data.”^ 

To  bring  the  specific  aims  of  this  research  into  sharp  focus,  a  concise  statement  of 
the  problem,  the  research  question  generated,  and  evaluative  criteria,  are  stated  below. 


^  The  word  “paradigm”  has  various  interpretations  (even  within  Kuhn’s  text)  to  include  “model,” 
“pattern,”  and  “worldview.”  Kuhn  however,  explicitly  prefers  to  define  the  concept  as  “like  an  accepted 
judicial  decision  in  the  common  law,  it  is  an  object  for  further  articulation  and  specification  under  new  or 
more  stringent  conditions.”  {The  Structure  of  Scientific  Revolutions,  23.)  This  definition  strikes  the  precise 
chord  for  the  contentions  advanced  in  this  thesis  and  is  adopted  for  the  purposes  of  this  study. 

®  Kuhn,  Thomas  S.,  7. 

^  Kuhn,  Thomas  S.,  x-xi. 
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A  PROBLEM,  A  QUESTION,  A  SEARCH  FOR  ANSWERS 


The  Problem 

As  outlined  above,  Commercial-off-the-shelf  (COTS)  acquisition 
practices  are  the  driver  for  system  procurement,  providing  business-model-based  tools 
that  are  not  suited  to  the  tasks  required  of  intelligence  analysts.  This  environment, 
referred  to  in  this  study  as  the  PC  Paradigm,  is  inspired  by  market  forces  and  is  not 
founded  upon  design  principles  that  support  analysis. 

The  Research  Question 

Could  the  current  personal  computer  environment  (characterized  as  the  PC 
Paradigm)  function  as  an  analysis  inhibitor,  rather  than  a  tool  to  facilitate  analysis? 

Evaluative  Criteria 

In  researching  the  areas  of  concern  stated  above,  the  following  criteria  will  guide 
the  evaluative  process: 

1 .  A  theoretical  basis  for  such  conditions  should  exist  (or  be  evident)  in  the 
literature,  which  may  be  exhibited  in  one  or  more  of  the  following  ways: 

a.  Dissatisfaction  with  the  current  paradigm. 

b.  Evidence  to  suggest  that  analyst  tasks  may  be  inadequately  supported. 

c.  Current  model  appears  to  violates/ignore  accepted  guidelines  within 
cognitive  science,  human-computer  interface  (HCI)  and  graphic  design. 

2.  Congruency  issues  will  be  apparent.  (Discussed  in  depth  in  Chapter  2.) 
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THE  PURSUIT  OF  ANALYTIC  RIGOR 


As  a  central  “core  competency”  of  the  intelligence  community  (IC),  considerable 
discussion  has  centered  on  what  constitutes  preferred  analytic  practice.  Such  debate  has 
been  ongoing  since  the  IC’s  inception  and  in  recent  years  has  been  the  personal  focus  of  a 
succession  of  Directors  of  Central  Intelligence  (DCI),  Seeking  to  bolster  analytic  rigor,  a 
concerted  effort  was  made  to  establish  “implementation  of  sound  analytic  standards  . . . 
ensuring  that  sufficient  attention  [was]  paid  to  cognitive  challenges  in  assessing  complex 
issues.”®  A  succinct  encapsulation  of  the  essence  of  this  school  of  thought  by  Deputy 
Secretary  of  Defense,  Paul  Wolfowitz  is  offered  below:^ 

Analysts  and  their  analysis  are  deemed  most  useful  when  they; 

•  Clarify  what  is  known  by  laying  out  the  evidence  and  pointing  to  cause-and-effect 
patterns. 

•  Carefully  structure  assumptions  and  argumentation  about  what  is  unknown  and 
unknowable. 

The  very  process  of  hypothesis-generation  and  seeking  “cause  and  effect  patterns” 
implies  that  mental  modeling  and  visualization  is  occurring.  This  contention  is  further 
supported  by  considerable  research  in  the  fields  of  cognitive  science,  graphic  display,  and 
human-computer  interface  (HCI)  technologies  that  suggest  that  considerable  overlap 
exists  in  theoretical  principles  and  applied  design  between  these  areas  and  intelligence 
analysis. 


®  Richards  J.  Heuer,  Psychology  of  Intelligence  Analysis  (Washington,  DC:  Center  for  the  Study  of 
Intelligence,  1999),  xv. 

^  Jack  Davis,  “The  Challenge  of  Managing  Uncertainty:  Paul  Wolfowitz  on  Intelligence-Policy 
Relations,”  Studies  in  Intelligence  39,  no.  5  (1996):  35. 
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The  brief  summary  above  is  offered  as  a  point  of  departure  for  the  further 
exploration  outlined  in  the  thesis  which  focuses  on  issues  that  exist  at  the  intersection  of 
cognitive  science,  graphic  display  visualization,  and  intelligence  analysis.  This  study 
does  not  seek  to  extend  the  debate  surrounding  the  issues  of  analytic  preferred  practice 
within  the  IC.  The  underlying  assumptions  that  frame  the  author’s  contentions  are 
defined  below. 

Tools  For  The  Analyst:  Unintended  Consequence? 

We  believe  the  analyst  of  tomorrow  must  be  freed  from  the  current  tool-centric  environment.  The 
analyst  must  be  allowed  to  focus  on  problem-solving,  not  tool  sets  or  questions  of  where  and  how 
data  is  obtained. 

-  F.J.  Hughes,  The  Art  and  Science  of  the  Process  of  Intelligence  Analysis 

In  the  pursuit  of  analysis,  as  in  any  endeavor,  tools  are  often  used  to  facilitate  the 
process.  Prior  to  the  widespread  use  of  automated  systems,  such  support  tools  typically 
consisted  of  a  variety  of  hardcopy  document  storage  and  retrieval  methods,  matrices, 
templates,  and  the  “corporate  memory”  of  the  intelligence  analyst.  As  information 
technologies  (IT)  matured,  the  evolution  of  computer- assisted  tools  that  were  employed 
in  intelligence  analysis  mirrored  that  of  the  private  sector.  Computing  resources 
underwent  a  dramatic  shift  from  centralized  mainframes  and  minicomputer  clusters  to 
stand-alone  and  networked  workstations.  The  personal  computer  (PC)  soon  became  the 
dominant  paradigm  for  the  infusion  of  information  technology. 

Tool  Utility:  The  Matter  of  Congruency.  The  shift  to  PC-based  platforms  was  a 
pivotal  juncture  that  marked  the  adoption  of  the  PC,  its  hardware  suite  and  interfaces  - 
including  its  fundamental  business-model  paradigm  —  as  the  de-facto  standard  for  DoD 
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automation.  Research  is  presented  that  suggests  that  the  market-driven  “one-size-fits-all” 
approach  of  the  PC  and  its  suitability  as  a  tool  to  support  intelligence  analysis  are  in 
conflict.  In  assessing  the  extent  and  potential  impact  of  such  conflict,  the  concept  of 
congruency  is  explored. 

Sometimes  referred  to  as  conformity,  the  concept  “defines  the  degree  of 
correspondence  between  the  mental  model  in  the  user's  mind  and  the  actual  system 
presentation.”  Simply  stated,  does  the  tool  facilitate  or  inhibit  the  work  it  is 
designed  to  assist?  This  concept  has  profound  implications  for  intelligence  analysis  and 
will  be  developed  and  explored  in  depth. 

Visualization  as  a  Tool.  Once  data  is  arrayed,  either  visually  or  in  a  mental 
representation,  the  mind  immediately  seeks  patterns  and  assigns  meaning  based  on  what 
it  “sees.”  The  effect  of  the  “presentation-as-interpretation”  is  considerable.  “A  visual 
image  not  only  organizes  the  data  at  hand  in  meaningful  structures,  but  is  also  an 
important  factor  guiding  the  analytical  development  of  a  solution.”'*  For  the  intelligence 
analyst,  data  representation  remains  secondary  to  analytical  depth,  and  the  information 
display  has  been  offered  as  a  tool  to  support  both  efforts. 

Some  studies  have  suggested  however,  that  “a  display  that  is  good  for  analysis  is 
one  that  is  bad  for  visualisation.”  This  is  an  intriguing  prospect  and  this  view,  along 


North  Atlantic  Treaty  Organization,  Visualisation  of  Massive  Military  Datasets:  Human 
Factors,  Applications,  and  Technologies,  RTO-TR-030  (Neuilly-Sur-Seine  Cedex,  France:  Research  and 
Technology  Organization,  May  2001),  59.  Cited  hereafter  as  NATO,  Visualisation  Study. 

”  Efraim  Fischbein,  Intuition  in  Science  and  Mathematics  (Dordrecht,  The  Netherlands;  Kluwer 
Academic  Publishers,  1987),  104. 

NATO,  Visualisation  Study,  84. 
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with  a  significant  body  of  work  supporting  the  visualization  discussion  is  examined 
in  depth  in  Chapter  2. 

It  is  important  to  discuss  visualization  in  terms  of  concept  and  desired  end-state, 
(i.e.  “What  do  we  want  to  achieve?”)  and  not  as  a  critique  of  specific  applications  or  to 
weigh  the  merits  of  particular  systems  and  schemas  under  development.  The  emphasis  is 
placed  on  process  and  goal  and  not  product  or  tool.  Bemsen  suggests  that  “the 
importance  of  a  vision  can  be  that  of  providing  a  model  within  which  we  think  and 

1 3 

create.  If  the  model  is  outdated,  thinking  becomes  unduly  constrained.” 

Let  us  then  define  visualization  as  a  conceptual  construct,  a  point  on  the  analysis 
continuum  rather  than  a  deliverable  product. 

The  PC  Paradigm:  A  Worldview  Challenged 

Both  the  private-sector  and  the  Department  of  Defense  have  come  to  rely  on 
commercial-off-the-shelf  (COTS)  products  to  form  their  technological  base  rather  than 
pursuing  in-house  development.  The  intent  is  well-meaning,  even  if  counterproductive. 
The  urge  to  supply  the  latest  desktop  technologies  to  the  workforce  is  commendable,  and 
standardization  greatly  facilitates  procurement  and  technical  support. 

Marketing  claims  aside,  however,  there  is  little  or  no  evidence  to  support  the 
contention  that  the  process  of  intelligence  analysis  has  been  augmented  or  enhanced  by 
the  presence  of  the  PC  on  every  desktop.  On  the  contrary,  it  has  forced  DoD  in  general 
(and  the  IC  in  particular)  to  function  within  a  business  model  exemplified  by  the  desktop 

Niels  Ole  Berasen,  “Natural  Human-Human  Computer  Interaction,”  in  Frontiers  of  Human- 
Centred  Computing,  On-Line  Communities  and  Virtual  Environments,  ed.  Earnshaw  and  others  (Berlin, 
Germany:  Springer  Verlag,  2001),  347. 
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metaphor.  The  reliance  on  this  widely-accepted  configuration  will  be  referred  to  as  the 


‘TC-Paradigmr 


What  Are  User-Interface  Metaphors  and  Why  Do  They  Matter?  Metaphor 
can  be  defined  in  the  linguistic  sense  as  “a  figure  of  speech  in  which  one  thing  is  likened 
to  another;  an  implied  comparison.”''^  A  user  interface  metaphor,  then,  serves  as  a 
representational  linkage  between  a  task  that  a  user  wishes  to  perform  and  its 
computational  counterpart. 

User  interface  metaphors  map  familiar  source  concepts  to  abstract,  computational 
target  domains.  For  example,  the  well-known  desktop  metaphor  introduced  by 
Xerox  and  popularized  by  Apple  allows  users  to  understand  the  operating  system 
of  a  computer  in  terms  of  the  objects  and  operations  at  an  office  desk.  Again  the 
metaphor  allows  us  to  understand  the  more  abstract  domain  (operations  on 
computer  files  and  directories)  in  terms  of  the  less  abstract,  familiar  operations  of 
handling  documents  and  folders  on  a  physical  desktop.'^ 

However,  further  complexity  can  arise.  “Is  the  desktop  metaphor  [a  single 
metaphor]  or  an  aggregate  of  many  (file,  folder,  clipboard,  etc.)?”'^  More  importantly,  is 
the  desktop  metaphor  the  best  choice  to  bridge  the  “abstract  domain”  of  the  system  and 
that  of  the  operations  familiar  to  the  intelligence  analyst? 

Metaphor  then,  in  the  sense  that  it  is  employed  in  system  design,  goes  well 
beyond  a  simplistic  linguistic  substitution.  As  noted  by  J.F.  Sowa,  “a  metaphor  does 
more  than  transfer  a  type  label;  it  transfers  an  entire  schema  or  cluster  of  schemata  to  a 


Webster’s  New  World  Dictionary  of  the  American  Language,  College  Ed.,  1968,  under  the  term 
“metaphor.” 

Werner  Kuhn,  “7±2  Questions  and  Answers  About  Metaphors  For  GIS  Interfaces,”  in  Cognitive 
Aspects  of  Human- Computer  Interaction  for  Geographic  Information  Systems  83,  ed.  T.L.  Nyerges  and 
others  (Dordrecht,  The  Netherlands:  Kluwer  Academic  Publishers,  1995),  114, 

Werner  Kuhn,  119. 
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new  type.”^^  This  powerful  concept  suggests  that  the  choice  of  metaphor  is  a  critical 
component  in  human -computer  interface  design  and  plays  an  active  role  in  shaping  the 
representation  of  knowledge.  An  examination  of  the  current  desktop  metaphor  for  “best 
cognitive  fit”  within  the  domain  of  intelligence  analysis  is  indicated  and  is  explored  in 
this  study. 

A  Critique  of  the  Current  Paradigm:  The  Desktop  Metaphor.  The  term 
desktop  metaphor  is  used  to  describe  the  current  user  interface  for  personal  computers 
and  most  workstations  that  evolved  from  the  research  of  Xerox  PARC  laboratories  in  the 
1970s.’®  Popularized  by  Apple  Computers  initially,  and  then  adopted  by  the  Microsoft 
DOSAVindows  personal  computer  environment,  it  has  become  the  de-facto  standard  for 
interface  design  throughout  private  industry,  and  subsequently,  the  military.  Based  on 
leveraging  the  familiarity  of  traditional  office  functions,  it  was  employed  as  a  means  for 
new  users  to  transition  from  “paper  documents  and  file  folders”  to  their  digital 
counterparts  of  word  processing  and  file  directories. 

While  a  highly  innovative,  if  not  brilliant,  step  forward  in  popularizing  the  PC  to  a 
worldwide  audience,  substantial  evidence  exists  that  the  thirty-years-old-plus  desktop 
metaphor  has  passed  beyond  relevance.  The  current  interface  represents  a  generic,  one- 
size-fits-all  approach  derived  entirely  from  a  business/office  model  and  geared  towards 
the  novice  user.  This  paradigm  is  market-driven  and  fails  to  take  into  account  the 

J.F.  Sowa,  Conceptual  Structures:  Information  Processing  in  Mind  and  Machine  (Reading,  MA: 
Addison- Wesley  Publishing  Co.,  1984),  27Q. 

Multiple  and  sometimes  contradictory  “histories”  of  the  origin  and  development  of  graphical 
user  interfaces  (GUI),  including  what  became  to  be  known  as  the  desktop  metaphor,  are  readily  available. 
Xerox  PARC  (Palo  Alto  Research  Center)  is  generally  credited  with  the  first  fielded  design  in  the  1970s, 
(although  commercially  unsuccessful)  which  was  later  revived/adapted  by  Apple  and  others. 
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increasing  sophistication  of  an  entire  generation  of  computer  users  that  are  not  products 
of  the  traditional  pre-automation  environment  and  cannot  be  defined  as  “novice-class.” 

POSITIONING  THE  RESEARCH 

This  study  was  undertaken  in  an  effort  to  question  whether  the  current  suite  of 
PC-based  tools  (collectively  termed  the  PC  Paradigm)  are  truly  suited  to  support  the 
stated  aims  of  the  intelligence  community.  These  aims  include  a  renewed  dedication  to 
analytic  rigor  that  advocates  the  use  of  structured  methodologies,  several  of  which  that 
emphasize  evidentiary  and  inferential  techniques. 

The  pursuit  of  such  “high-order  analysis”  relies  heavily  on  visualization,  mental 
modeling,  and  knowledge  representations  that  have  important  implications  in  terms  of 
human -computer  interface  and  cognitive  process.  “The  usability  of  such  human- 
computer  interfaces  depends  on  how  the  cognitive  abilities  of  the  user  are  either 
facilitated  or  stressed  by  the  interface.”*® 

As  pointed  out  in  the  opening  section  of  this  thesis,  the  role  of  Commercial-off- 
the-Shelf  (COTS)  technologies  is  tightly  linked  to  this  discussion.  COTS  approaches  are 
specifically  examined  for  their  applicability  to  the  problems  faced  by  the  intelligence 
analyst.  This  examination  is  informed  by  an  interdisciplinary  review  of  both  theoretical 
and  applied  design  considerations  using  widely-accepted  models  in  various  related  fields. 


Kent  L.  Norman,  Interface  Apparency  and  Manipulatability:  Cognitive  Gateways  Through  The 
Spatial  Visualization  Barrier  In  Computer-Based  Technologies,  Monograph,  University  of  Maryland 
(College  Park,  MD;  N.p.,  n.d.)  2. 
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This  study  is  unique  in  that  it  illuminates  a  growing  body  of  literature  that  brings 
to  bear  diverse  perspectives  from  disciplines  outside  the  normal  scope  of  the  intelligence 
community,  and  seeks  to  provide  a  synthesis  of  multidisciplinary  research. 

ASSUMPTIONS 

The  Domain  of  Intelligence  Analysis 

This  study  is  oriented  towards  the  professional  intelligence  community  and  as 
such,  certain  assumptions  will  frame  the  discussion.  First  and  foremost  is  an  agreement 
upon  the  goals  and  intentions  of  intelligence  analysis  as  set  forth  in  the  intelligence 
literature  from  the  community’s  inception  to  the  present  day.  While  debate  is  certainly 
acknowledged  and  the  nuances  have  been  the  subject  of  countless  papers,  articles,  and 
theses,  the  general  “philosophy”  of  the  intelligence  cycle  has  remained  as  a  fundamental 
principle.  Within  this  familiar  cycle  of  “requirement,”  “collection,”  “production,”  and 
“dissemination”  lies  the  implied  task  of  analysis  -  the  raison  d’etre  of  our  profession. 

For  the  purposes  of  this  paper,  analysis  will  be  assumed  as  the  driving  imperative 
and  the  support  of  analysis  as  the  purpose  of  all  tools  (automated  or  not)  employed  in  its 
pursuit.  Further,  for  the  sake  of  expediency  and  based  on  academic  principle,  the 
preferred  method  of  performing  analysis  is  grounded  in  the  prineiples  espoused  by  the 
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curriculum  of  the  Joint  Military  Intelligence  College,  and  encompassing  the  suggested 
guidance  and  models  of  MacEachin  ,  Heuer  ,  Toulmin  ,  Schum  ,  and  Hughes. 

These  models  of  critical  thinking  move  beyond  the  simplistic  and  intuitive, 
representing  high-order  cognitive  function  and  the  ability  to  replicate  and  illustrate  the 
analytical  process.  They  include  Analysis  of  Competing  Hypotheses  (ACH)  methods,  the 
use  of  evidence  and  inference,  argumentation,  and  other  techniques  employing 
“imaginative  and  critical  reasoning.”^^  Acknowledging  the  background  of  the  intended 
audience,  it  is  expected  that  the  reader  is  familiar  with  the  intelligence  domain  and 
minimal  time  will  be  spent  on  elaborating  on  the  theme  except  for  cases  of  clarification. 

The  Applicability  of  Constructs  Common  to  the  Sciences 

It  is  the  author’s  belief  that  theoretical  concepts  that  apply  to  what  Kuhn^^  refers 
to  as  "“normal  science”  have  application  to  the  problems  facing  the  intelligence 
community.  Intelligence  as  a  profession  has  much  in  common  with  the  sciences;  utilizing 


Douglas  J,  MacEachin,  former  Deputy  Director  of  Intelligence  (1993-1996)  was  particularly 
noted  for  his  strong  beliefs  in  analytic  process  and  critical  thinking.  MacEachin  sought  to  incorporate 
tradecraft  standards  for  analysts  and  advocated  structured  argumentation  through  “linchpin  analysis.” 

A  brief,  but  informative  assessment  of  the  former  DDEs  impact  is  found  in  Heuer’s  Psychology  of 
Intelligence  Analysis,  pp.  xv-xvii,  as  cited  in  the  bibliography. 

Richards  J.  Heuer,  as  cited  in  the  bibliography. 

Stephen  E.  Toulmin,  as  cited  in  the  bibliography. 

Dr.  David  A.  Schum,  Professor,  George  Mason  University,  as  cited  in  the  bibliography. 

^  Francis  J.  Hughes,  Professor,  Joint  Military  Intelligence  College,  as  cited  in  the  bibliography. 

Francis  J.  Hughes,  The  Art  and  Science  of  the  Process  of  Intelligence  Analysis,  course  material 
handout  developed  for  the  analysis  curriculum  (Washington,  DC:  Joint  Military  Intelligence  College, 
September  2002):  2. 

Thomas  S.  Kuhn,  10. 
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both  qualitative  and  quantitative  methodologies,  a  belief  in  scientific  method  and  the 
advocacy  of  the  pursuit  of  excellence  in  research. 

Intelligence  analysis  is  inherently  interdisciplinary  and  often  draws  upon 
knowledge  and  skills  from  multiple  knowledge  domains.  Such  cross-fertilization  of  ideas 
is  to  be  encouraged  and  should  be  expanded  upon  to  avoid  the  tendency  to  limit  solution 
possibilities  to  those  within  immediate  grasp.  This  paper  seeks  to  offer  a  linkage  to  ideas 
and  concepts  that  are  usually  found  outside  the  normal  scope  of  the  intelligence  analyst. 

Intangibles 

Intelligence  analysis,  cognitive  science,  and  human -computer  interface  design 
have  all  been  characterized  as  “art”  as  well  as  science.  Discussion  of  theoretical  and 
applied  concepts  in  these  and  related  fields  must  contend  with  sometimes  fractious 
schools  of  thought  and  can  descend  into  endless  debate.  Recognizing  the  potential  for 
subjective  division,  every  attempt  has  been  made  to  remain  on  ‘solid  ground”  and  avoid 
overly  controversial  positions  within  the  respective  fields  of  research  and  instead  to 
illustrate  concepts  with  generally-accepted  examples. 

SCOPE  CONSIDERATIONS 

Synthesis,  Reduction,  and  Focus 

In  attempting  a  multi-disciplinary  fusion  of  research  findings  spanning  a  rich  and 
sometimes  contentious  collection  of  sources,  a  certain  amount  of  reductionism  is 
required.  The  focus  remains  on  showing  how  multiple  lines  of  inquiry,  particularly 
within  the  cognitive  science  and  interface-design  schools  of  thought,  can  reveal  a 
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convergence  of  design  theory  that  is  directly  applicable  to  analytic  practice.  This  paper  is 
not  intended  as  an  exhaustive  treatment  of  each  contributing  field  of  study,  but  rather  as 
an  illumination  of  shortcomings  in  the  accepted  PC  Paradigm  and  to  suggest  solutions 
based  on  grounded  research  that  reside  in  areas  of  study  outside  the  usual  range  of 
interest  for  the  intelligence  analyst. 


On  Advanced  Graphics  Workstations 

It  is  acknowledged  that  specialized  workstations,  generally  “high-end”  and 
capable  of  sophisticated  image  manipulation  have  been  in  use  within  DoD  for  some  time. 
Currently,  these  workstations  are  most  widely  found  in  the  geospatial  information 
systems  (GIS)  and  imagery  analysis  (IMINT)  sectors,  although  a  very  active  R&D  effort 
continues  DoD-wide  with  extensive  contractor  involvement.  Intriguing  potential  aside, 
the  reality  is  that  comparatively  few  members  of  the  intelligence  analysis  community  use 
such  specialized  workstations  in  their  daily  duties  (GIS/IMINT  workstation  exception 
noted). 


An  excerpt  from  a  U.S.  Army- sponsored  research  project  commissioned  to 

■97 

prototype  one  such  system  is  illustrative  of  the  pragmatics  involved  in  the  actual 
fielding: 

Although  exceptionally  good  graphics  capabilities  are  supported  by  Unix 
machines  such  as  the  Silicon  Graphics,  Inc.  IRIS  series,  the  relatively  high 
price/performance  ratios  eliminated  them  from  further  consideration  as  potential 
hosts  ...  we  thus  favored  the  Macintosh  OS  and  DOSAVindows  environments. 
Although  the  former  provides  superior  graphics  tools,  we  selected  the  latter 


Greg  Zacharias  and  others,  VIEW:  Visualization  and  Interactive  Elicitation  Workstation  -  A 
Tool  For  Representing  the  Commander’s  Mental  Model  of  the  Battlefield,  Monograph,  U.S.  Army 
Research  Institute  for  the  Behavioral  and  Social  Sciences  (Cambridge,  MA:  Charles  River  Analytics, 
December,  1996),  5. 
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because  of  the  much  larger  installed  base. .  .and  the  greater  likelihood  of 

integration/networking  with  existing  Army  systems. 

In  practical  terms,  advanced  workstations  are  available  and  provide  specialized 
tools  to  a  certain  subset  of  the  analyst  community,  while  the  desktop  PC  and  the  laptop 
remain  the  primary  tool  for  the  majority.  This  study  does  not  address  the  high-end, 
specialized  workstation  segment  except  in  the  cases  of  desirable  design  principles  that 
generally  apply. 

On  Specialized  Commercial  Software 

As  in  the  case  of  specialized  workstations  described  above,  a  variety  of 
commercial  software  packages  are  available  to  support  visualization  and  data 
exploration.  Often  designed  to  augment  complex  quantitative  datasets,  popular  examples 
include  Mathematical^,  MatLab™,  stnd  Analytica™ among  others. 

Rather  than  characterizing  these  types  of  packages  as  contrafactual  examples  of 
the  PC  Paradigm,  the  author  submits  that  such  specialized  packages  were  created  in 
response  to  a  recognized  need  for  analytical  support  that  was  not  being  met  by  the 
existing  product  offerings  in  their  respective  researcfi  domains.  Such  a  requirement  is 
keenly  felt  within  the  field  of  intelligence  analysis  as  well,  and  the  lack  of  effective 
fielded  solutions  is  evident. 


^  Mathematica  is  a  trademark  of  Wolfram  Research  found  at  www.wolfram.com;  MatLab  is  a 
trademark  of  Math  works  found  at  www,  mathworks.com;  Analytica  is  a  trademark  of  Lumina  Decision 
Systems,  found  at  www.lumina.com. 
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On  the  Advanced  Research  and  Development  Activity  (ARDA) 

The  need  for  revolutionary  research  programs  within  the  intelligence  community 
is  recognized  and  sponsored  by  the  Advanced  Research  and  Development  Activity 
(ARDA),  located  in  Fort  Meade,  Maryland^^.  ARDA’s  focus  on  high-risk,  high-payoff 
research  is  structured  to  leverage  multi-disciplinary  expertise  against  existing  or 
emerging  problems  and  assist  the  transfer  of  potential  solutions  to  the  IC  technology 
centers. 

In  researching  this  study,  the  author  has  reviewed  several  proposals  and 
presentations  outlined  in  the  workshop  and  program  review  notes  for  one  of  the  ARDA 
projects  known  as  the  Geospatial  Intelligence  Information  Visualization  (GFVIS) 
program.  Highly  intriguing  research  initiatives  are  in  progress  that  address  such  issues 
as  visual  metaphor,  iconography  approaches,  tailorable  visualizations,  and  other  design 
concepts  intended  to  take  visualization  support  for  the  analyst  to  the  next  level. 

This  notable  effort  directly  emphasizes  the  need  for  multidisciplinary  (or 
interdomain)  approaches  such  as  are  advocated  in  this  study.  The  visionary  research 
supported  by  ARDA,  however,  remains  speculative  and  holds  promise  for  the  future.  For 
the  purposes  of  this  thesis,  an  assessment  of  the  many  ARDA  research  projects  remains 
outside  the  intended  scope. 


^  For  more  information,  the  ARDA  web  site  can  be  accessed  at  http://  www.ic-arda.org.  The 
characterization  of  the  ARDA  mission  cited  herein  was  derived  from  this  unclassified  site. 

Department  of  Defense,  Advanced  Research  and  Development  Activity,  GPVIS  12-Month 
Workshop  Notes,  31  October  -  1  November  2001  (McLean,  VA;  The  MITRE  Corporation)  and  GPVIS 
Phase  I  Program  Review,  16-18  October  2002  (McLean,  VA:  The  MITRE  Corporation). 
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On  Alternative  Interfaces 


Cutting-edge  research  continues  on  alternative  interfaces  designed  to  facilitate 
human  computer  interaction  through  non-traditional  methods.  These  alternatives  include 
Virtual  Reality  (VR),  immersive  techniques,  haptic  (sense  of  touch)  interfaces,  data 
sonification  (use  of  sound  context)  and  others.  While  such  alternatives  are 
intriguing,  this  study  is  not  intended  as  a  platform  to  champion  a  particular  conceptual 
interface.  A  particularly  thorough  treatment  of  near-term  and  fielded  alternate  interfaces 
is  found  in  the  NATO  study,  Visualisation  of  Massive  Military  Datasets:  Human 
Factors,  Applications,  and  Technologies.^’ 

The  Data  Overload  Problem 

The  concept  of  data  overload  will  be  assumed  as  widespread  and  persistent.  This 
well-documented  phenomenon  is  not  confined  to  the  intelligence  community,  and  is  an 
active  subject  of  research  in  all  sectors.  Proposed  solutions  to  this  problem  are  not 
discussed,  but  the  impact  on  the  analyst  of  the  exponential  increase  in  data  streams  is 
considered.  This  increase  is  particularly  evident  in  the  recent  proliferation  of  sensor- 
types,  including  availability  of  commercial  remote-sensing  data,  UAV  platforms,  and 
unattended  devices. 


NATO,  Visualisation  Study. 
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THESIS  RESEARCH  OVERVIEW 


Based  on  the  interdisciplinary  nature  of  this  paper,  an  expanded  literature  review 
and  discussion  follows  in  Chapter  2.  To  facilitate  the  arrangement  of  diverse  data  the 
review  has  been  presented  in  a  series  of  ten  sections  that  address  the  research  question 
with  applicable  and  relevant  commentary  arranged  by  general  field  of  study  or  conceptual 
point  of  view.  These  sections  include: 

•  A  Discussion  of  Paradigm,  Theory,  and  Scientific  Endeavor 

•  A  Look  at  the  Current  Model 

•  Intelligence  Analysis  as  Cognitive  Process 

•  Applicable  Concepts  of  Psychology  and  Cognitive  Science 

•  Visualization  and  Mental  Models 

•  Tenets  of  Graphic  Design 

•  Human-Computer  Interface  (HCI)  Design  Considerations 

•  European/NATO  Perspective 

•  Department  of  Defense  (DoD)  Guidance 

•  Cognitive  Task  Analysis  (CTA)  Methodologies 

Research  methodology  and  findings  are  presented  in  Chapters  3  and  4,  while 
conclusions  and  recommendations  for  future  study  comprise  the  final  section.  Chapter  5. 
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CHAPTER  2 


A  MULTI-DISCIPLINARY  REVIEW:  RELEVANT  LITERATURE 

AND  DISCUSSION 


This  review  presents  an  exhaustive  survey  of  cross-disciplinary  literature  that 
encompasses  applicable  research  extant  in  the  fields  of  intelligence  analysis,  cognitive 
science,  data  visualization,  graphics  design,  and  human-computer  interface  (HCI)  design. 
While  the  available  literature  in  multiple  fields  is  considerable,  a  focused  survey  reveals 
key  papers  and  texts  that  are  particularly  suited  to  address  the  issues  that  exist  at  this 
intersection  of  disciplines  and  necessitates  a  literature  review  of  greater  expanse  than 
generally  called  for  in  studies  that  draw  primarily  from  a  single  knowledge  domain. 

Valuable  applicable  data  is  sometimes  fragmented,  appearing  in  a  wide  range  of 
both  military  and  non-DoD  publications  and  likely  to  escape  notice  by  the  analyst 
community.  The  intent  is  to  illuminate  important  theoretical  and  applied  research  and 
achieve  a  synthesis  among  the  disparate  bodies  of  research.  While  effort  has  been  made 
to  present  relevant  research  under  appropriate  categories  as  outlined  below,  the 
interdisciplinary  nature  of  the  material  exhibits  considerable  overlap  and  often  resists 
attempts  at  strict  categorization.  This  is  however,  a  desirable  intersection,  and  serves  to 
illustrate  the  interlocking  and  mutually  supportive  nature  of  the  research. 

A  Discussion  of  Paradigm,  Theory,  and  Scientific  Endeavor 

Any  discussion  of  theoretical  constructs,  paradigms,  and  conceptualizations  can 
easily  encompass  idealized  and  philosophical  notions.  These  discussions,  while 
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interesting,  can  stray  from  the  focus  of  the  subject  at  hand  and  as  such,  will  be  limited  to 
bringing  out  those  points  with  direct  application. 

Kuhn’ s  discussion  of  paradigm  is  widely  referenced  throughout  the  literature 
and  is  key  to  understanding  how  “worldview”  influences  cycles  of  scientific  progress. 
This  thesis  considers  Kuhn’s  definition  of  normal  science^^  and  his  view  of  periodic 
cycles  of  revolutionary  progress,  as  directly  analogous  to  the  workings  of  the  intelligence 
community.  Within  the  IC,  “normal  analysis”  can  be  viewed  as  a  collection  of  accepted 
practices  interspersed  with  periods  of  rapid  change.  Drawing  upon  another  model  from 
the  sciences,  this  fitful  progress  proceeds  in  a  fashion  not  unlike  that  of  the  punctuated 
equilibrium  theories  advanced  by  Gould, i.e.  long  periods  of  relative  stasis  punctuated 
with  periods  of  rapid  and  dramatic  upheaval  and  change.  There  is  a  general  consensus 
that  the  events  of  September  1 1**’,  2001  marked  just  such  an  epoch,  and  may  serve  as  the 
driving  imperative  to  further  “paradigm  shifts”  within  the  IC. 

When  viewed  in  this  manner,  Kuhn’s  observations  on  mindset/worldview/ 
paradigm  can  offer  insight  into  “intelligence  crises”  that  seem  to  recur.  One  might  well 
substitute  the  world  "'intelligence^' ,  as  this  author  has  done,  in  place  of  "science"  in 
Kuhn’s  pronouncement  below 

When  ...  the  profession  can  no  longer  evade  anomalies  that  subvert  the  existing 
tradition  of  scientific  [intelligence]  practice  -  then  begin  the  extraordinary 
investigations  that  lead  the  profession  at  last  to  a  new  set  of  commitments,  a  new 

Thomas  S.  Kuhn,  23. 

Thomas  S.  Kuhn,  10.  Kuhn  defines  normal  science  as  “research  firmly  based  upon  one  or  more 
past  scientific  achievements,  achievements  that  some  particular  scientific  community  acknowledges  for  a 
time  as  supplying  the  foundation  for  its  further  practice.” 

Stephen  Jay  Gould,  The  Structure  of  Evolutionary  Theory  (Cambridge,  MA:  The  Belknap  Press 
of  the  Harvard  University  Press,  2002),  765-784. 

Thomas  S.  Kuhn,  6. 
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basis  for  the  practice  of  science  [intelligence^.  The  extraordinary  episodes  in 
which  that  shift  of  professional  commitments  occurs  are  the  ones  known  ...  as 
scientific  [intelligence]  revolutions.  They  are  the  tradition-shattering 
complements  to  the  tradition-bound  activity  of  normal  science  [intelligence 
practice] . 

The  relevance  of  paradigm,  elaborately  and  eloquently  set  forth  in  Kuhn’s  text, 

cannot  be  overstated.  Kuhn  remarks  that  “[established  viewpoints  and  preconceptions] 

...  can  operate  as  a  sort  of  restriction  [that  can]  bound  the  admissible  solutions  to 

theoretical  problems.”^^  In  intelligence  analysis,  as  in  science,  it  is  imperative  that 

the  “admissible  solutions”  remain  “unbound”  until  either  refuted  or  confirmed. 

Formalizing  this  process  by  requiring  the  representation  of  the  reasoning  chains  involved 

in  analysis  is  a  primary  goal  of  research  and  applications  advocated  by  Schum, 

MacEachin,  Heuer,  Hughes,  and  others  as  illustrated  below. 

Dreyfuss  points  out  that  the  “reigning  conceptual  framework  implicitly  guides 

research  just  as  the  perceptual  field  guides  our  perception  of  objects.”^^  Without 

invoking  the  concept  of  paradigm  explicitly,  Dreyfuss  makes  note  of  the  negative  aspects 

of  becoming  attached  or  anchored  to  a  particular  systemic  view  by  stating, 

that  one  must  not  become  so  fascinated  with  the  formalized  aspects  of  a  subject 
that  one  forgets  the  significant  questions  which  originally  gave  rise  to  the 
research,  nor  should  one  be  so  eager  for  experimental  results  that  one  continues  to 
use  old  techniques  just  because  they  work,  when  they  have  ceased  to  lead  to  new 
insights. 


Thomas  S.  Kuhn,  39. 


”  Hubert  L.  Dreyfuss,  Whal  Computers  Still  Can 't  Do: 
(Cambridge,  MA:  The  MIT  Press,  1992):  279. 


A  Critique  of  Artificial  Reason 


Dreyfuss,  233. 
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This  paper  considers  paradigm  conflict  to  be  a  key  contributing  factor  to 
visualization  channeling  that  may  restrict  the  solution  sets  available  (or  apparent  to)  the 
analyst. 


A  Look  at  the  Current  Model 

For  many  people,  the  desktop  represents  a  permanent,  perceptually  valid  contribution  to 
computer  interface  design.  ‘Like  many  other  brilliant  ideas,’  Steven  Levy  writes  of  the 
desktop  model  as  it  emerged  from  PARC,  ‘once  introduced  it  is  unimaginable  to  conceive 
of  working  without  it.’ 

David  Gelemter,  Machine  Beauty 

The  ingrained  familiarity  of  what  is  sometimes  derisively  termed  the  “WIMP” 
interface  (Windows,  Icons,  Menus,  Pointers),  continues  to  undergird  the  current 
paradigm.  Considerable  debate  exists  however,  on  whether  the  desktop  metaphor  that 
ushered  in  the  rapid  acceptance  of  the  personal  computer  remains  relevant. 

Center  and  Nielsen  point  out  that  “although  the  use  of  metaphor  may  ease 
learning  for  the  computer  novice,  it  can  also  cripple  the  interface  with  irrelevant 
limitations  and  blind  the  designer  to  new  [and  more  appropriate]  paradigms.”^^ 

David  Gelemter  characterizes  such  limitations  as  exhibiting  the  effect  of  paradigm  drag 
which  “comes  in  when  the  electronic  version  is  cramped  by  the  limitations  of  the  physical 
one.”'**^  Gelemter  further  observes  that  entities  such  as  file,  folder,  and  desktop  are 
“earthbound-metaphors, . . .  ideas  [that]  are  patterned  so  closely  on  preexisting  physical 
models  that  the  software  is  seriously  constrained.”''^* 

Don  Gentner  and  Jakob  Nielson,  “The  Anti-Mac  Interface,”  Communications  of  the  ACM  39, 
no,  8  (August  1996);  71. 

’***  David  Gelemter,  Machine  Beauty:  Elegance  and  the  Heart  of  Technology  (New  York:  Basic 
Books,  1998),  89. 

Gelemter,  64. 
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The  desktop  metaphor  and  its  office-based  conceptual  structure  is  not  the  sole 

source  of  concern.  The  familiar  use  of  “windows”  as  a  means  of  managing  multiple 

streams  of  data  creates  a  cognitive  loading  effect  that  must  be  taken  into  consideration. 

Multi-windowing  environments,  while  common  and  indeed  useful,  place  demands  on  the 

user  to  optimize  the  display  configuration.  When  this  effect  is  ignored  as  part  of  the 

design  process,  the  end  result  often  results  in  clutter  and  distraction  rather  than  enhancing 

information  display,  as  observed  in  the  following  cognitive  study 

One  pitfall  is  to  rely  on  the  flexibility  of  multi-windowing  systems  to  substitute 
for  detailed  analysis  of  information  display  requirements.  The  premise  is  that 
users  can  call  up  and  configure  many  views  in  parallel,  tailoring  the  display  to 
their  particular  needs  at  a  given  point  in  time.  However,  this  strong  reliance  on 
the  skills  of  users  to  configure  displays  optimally  represents  a  failure  of  design. 
Handing  off  too  much  of  this  task  to  users  creates  extra  data-management  tasks 
that  shift  the  burden  of  task  analysis  and  display  configuration  onto  users.  The 
user’s  limited  attention  resources  are  shifted  to  the  interface  in  order  to  identify 
desired  data,  navigate  to  the  necessary  location  in  the  display  space,  and  configure 
coordinated  views.  The  CRT  can  become  cluttered  with  windows,  causing 
problems  in  determining  where  to  focus  attention  and  making  it  easy  for  the  user 
to  miss  new  events.  These  navigation  and  interface-management  activities  place 
high  demands  on  user  memory  rather  than  capitalize  on  the  interface  as  an 
external  representation  aid. 

Similar  observations  are  made  by  Medyckyj-Scott  who  notes  that, 

“although  multiple  windows  increase  the  perceived  viewing  space,  they  may  not 
necessarily  increase  the  visual  scope  if  the  user  is  unable  to  see  a  relationship 
which  spans  the  display  or  is  unable  to  locate  which  window  holds  pertinent 
data.”^^ 

In  addition  to  the  discussion  of  desktop  metaphor  and  the  windowing 
environment,  a  third  aspect  of  the  current  paradigm  will  be  examined  —  the  office-based 
software  suite  itself.  Specialized  software  aside,  the  typical  commercial  software 

Emilie  M.  Roth  and  others.  Cognitive  Engineering:  Issues  in  User-Centered  Systems  Design, 
Monograph,  Roth  Cognitive  Engineering  (Brookline,  MA;  N.p,,  1994),  4-5. 

D,  Medyckyj-Scott,  “Visualization  and  Human-Computer  Interaction  in  GIS,”  in  Visualization 
in  Geographic  Information  Systems,  ed.  Hilary  M.  Hearnshaw  and  David  J,  Unwin  (New  York:  John  Wiley 
and  Sons,  1994),  205. 
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package  consists  of  a  suite  of  database,  word  processing,  and  spreadsheet  applications. 
Microsoft  Office™  is  a  popular  and  typical  configuration. 

The  author  contends  that  such  software  tools  introduce  an  inherent  cognitive 
influence  that  shapes  the  thinking  of  the  user  by  channelizing  or  limiting  “admissible 
solutions.”  To  illustrate  by  means  of  a  simplified  analogy,  the  example  of  the  rapid 
acceptance  of  the  spreadsheet  program  as  the  near-universal  model  of  financial 
and  accounting  analysis  is  offered, 

Spreadsheet  software  was  one  of  the  early  “killer  app”'*'^  market  successes  that 
helped  provide  much  of  the  momentum  during  the  PC’s  introduction.  The  familiar 
column  and  row  representation  of  data  “cells”  offered  flexible  attribute  assignment  and 
rapid  recalculation  and  became  the  de-facto  standard  of  financial  analysis.  One  could 
safely  assume  that  if  asked  to  perform  nearly  any  type  of  financial  analysis  today,  a 
spreadsheet  program  would  almost  universally  be  consulted  to  array  and  compare  the 
data. 

At  this  point  in  the  discussion,  it  could  be  argued  that  the  spreadsheet  model  has 
implicitly  defined  how  the  financial  analysis  will  be  conducted  and  explicitly  defined  how 
the  resultant  answers  will  be  represented  to  the  user.  Expressed  in  a  different  manner,  the 
representation  of  the  data  and  its  variable  relationships  is  determined  by  the  spreadsheet, 
and  thus  shapes  the  user’s  visualization  of  the  dataset  and  its  mental  model.  The 
conceptualization  is  thereby  channelized  and  delimited  by  the  spreadsheet  construct. 

Bill  Gates,  The  Road  Ahead  (New  York;  Viking/Penguin  Group,  1995),  68-69.  The  Verm 
“killer  application”  (or  more  commonly  “killer  app”)  is  widely  used  to  connote  a  wildly  successful  product, 
generally  referring  to  a  commercial  software  application.  Gates  describes  it  as  a  “use  of  technology  so 
attractive  to  consumers  that  it  fuels  market  forces  and  makes  an  invention  all  but  indispensable,  even  if  it 
wasn’t  anticipated  by  the  inventor,”  Early  “killer  apps”  in  the  software  realm  included  WordStar™, 
dBase'*'”,  and  Visicalc^^”. 
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As  noted  by  Roth  and  others'*^,  “it  is  a  fundamental  scientific  finding  that  how 

a  problem  is  represented  affects  the  cognitive  work  that  is  needed  to  solve  that 

problem,  referred  to  as  the  representation  effect.”  Further  research  on  this  effect, 

specifically  in  regards  to  intelligence  problems,  is  warranted. 

Despite  the  likelihood  of  “channelization,”  the  value  (and  continued  commercial 

success)  of  the  spreadsheet  is  found  in  its  particular  suitability  to  the  domain  of  finance 

and  accounting.  It  is  this  domain  relevance  however,  that  is  lacking  in  business-model 

tool  sets  when  applied  to  the  problems  of  intelligence  analysis. 

Nobody  would  argue  that  an  accounting  program  like  Visicalc  ‘thinks’  about 
business,  but  it  is  a  vital  tool  because  of  the  clear  and  appropriate 
correspondence  between  its  domain  and  the  activities  that  generate  the 
commercial  world.  Another  widespread  example  is  ‘word  processing.’  Its 
domain  is  the  superficial  stuff  of  language  -  letters  and  punctuation  marks,  words, 
sentences,  and  paragraphs.  A  ‘word  processor’  does  not  understand  language,  but 
can  be  used  to  manipulate  text  structures  that  have  meaning  to  those  who  create 
and  read  them.  The  impact  comes  not  because  the  programs  are  ‘smart’  but 
because  they  let  people  operate  effectively  in  a  systematic  domain  that  is  relevant 
to  human  work."^®  (italics  mine) 

A  discussion  of  relevant  domain  issues  encountered  within  intelligence  analysis  is 
presented  in  the  following  section. 


Intelligence  Analysis  as  Cognitive  Process 

Although  geared  for  the  intelligence  praetitioner,  the  limits  of  discussion  are 
outlined  in  the  assumptions  provided  in  Chapter  1.  The  intent  is  not  to  fully  describe  or 
debate  the  relative  merits  of  analytical  methodologies,  but  to  introduce  the  recognized 

Emilie  Roth  and  others,  6. 

^  Terry  Winograd  and  Fernando  Flores,  Understanding  Computers  and  Cognition:  A  New 
Foundation  For  Design  (Norwood,  NJ:  Ablex  Publishing  Corporation,  1986),  175. 
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cognitive  aspects  of  analysis  that  should  be  addressed  during  considerations  of  optimum 
design.  This  section  introduces  some  of  the  key  underpinnings  in  analytical  thought,  and 
examines  the  findings  of  a  DoD  study  that  specifically  addresses  cognitive  processes  in 
intelligence  analysis.  Finally,  unique  aspects  of  the  intelligence  domain  that  apply  to  the 
discussion  are  presented. 


Data  vs  Analysis.  Intelligence  analysis  is  concerned  with  problem-solving,  not 

merely  the  accumulation,  manipulation,  and  presentation,  of  data.  As  noted  by  Grabo: 

“facts  don’t  speak  for  themselves ...  If  the  sole  function  of  intelligence  was  to 
compile  ‘facts,’  there  would  be  little  need  for  analysts  of  any  type.  The 
intelligence  process  would  consist  almost  entirely  of  collection  of  raw  data  which 
would  be  evaluated  for  accuracy  but  then  passed  on  without  further  comment  or 
analysis  to  the  policy  official.'*^ 

Traditional  reliance  on  order-of-battle  summaries,  comparative  force  assessments, 
weapon  and  equipment  capabilities  studies,  and  unit  locations  has  often  belied  the 
contention  that  analysis  is  indeed  more  than  data  representation.  It  should  be  noted  that  it 
is  not  coincidental  that  our  intelligence  collection  platforms  are  optimized  to  gather  such 
information.  More  importantly,  the  existing  automated  tools  generally  available  to  the 
analyst  support  just  this  type  of  data  representation,  while  the  analytic  component  is 
largely  left  unaided. 

The  literature  suggests  that  these  different  modes  of  intelligence  analysis 
have  long  been  recognized,  even  if  not  reflected  in  our  available  tool  sets.  Heuer 
distinguished  between  data-driven  analysis  and  concept-driven  analysis‘s  as  early  as 

Cynthia  M.  Grabo,  Anticipating  Surprise:  Analysis  for  Strategic  Warning,  (Washington,  DC: 
Joint  Military  Intelligence  College,  December  2002),  133. 

Heuer,  59.  Heuer  mentions  in  a  footnote  on  page  5 1  that  this  material  originally  appeared  in 
Studies  in  Intelligence  23,  no.  1  (Spring  1979)  and  also  appeared  in  modified  form  in  The  Bureaucrat  8, 
1979. 
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1979,  and  the  findings  of  a  contemporary  DoD  study  (discussed  below)  share  similar 
views. 

A  closer  look  at  the  works  of  Richards  J.  Heuer  is  particularly  illustrative  in  this 
discussion.  The  mere  fact  that  Heuer’ s  collection  of  essays  was  entitled  The  Psychology 
of  Intelligence  Analysis  is  evidence  of  an  early  recognition  of  the  centrality  of  cognitive 
factors  in  the  conduct  of  what  is  collectively  termed  “intelligence  analysis.”  The  majority 
of  the  papers  were  written  between  1978-1986  (although  published  in  book  form  in 
1999),  and  thus  preceded  the  widespread  introduction  of  the  personal  computer.  His 
papers  underscored  the  point  that  psychological  and  cognitive  principles  are  intrinsically 
linked  to  intelligence  analysis,  and  to  its  detriment,  often  overlooked.  Convinced  that 
“analytical  aids”  were  necessary,  Heuer  observed  that  “tools  and  techniques  that  gear  the 
analyst’s  mind  to  apply  higher  levels  of  critical  thinking  can  substantially  improve 
analysis  on  complex  issues  on  which  information  is  incomplete,  ambiguous,  and  often 
deliberately  distorted.”'^^ 

One  influential  method  of  providing  such  tools  to  augment  the  critical  thinking 
process  is  found  in  the  works  of  Dr.  David  Schum.^°  A  central  figure  in  matters 
involving  evidential  and  inferential  techniques  as  applied  to  intelligence  analysis,  his 
two-volume  work  is  an  erudite  treatise  on  the  subject.  It  brings  together  the  elements  of 
jurisprudence,  legal  argumentation  and  classical  logic  into  a  comprehensive  synthesis 
directed  at  aiding  the  intelligence  analysis  in  hypothesis  evaluation  and  the  drawing  of 


Heuer,  xviii. 

David  A.  Schum,  Evidence  and  Inference  For  The  Intelligence  Analyst,  Volumes  1  and  2, 
(Lanham,  MD:  University  Press  of  America,  Inc.,  1987). 
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defensible  conclusions.  In  agreement,  Hughes^*  contends  that  the  concept  of  discovery- 
proof-choice  are  the  core  activities  of  intelligence  analysis. 

Of  particular  interest,  Schum  poses  the  following  conundrum:  “Two  astute, 
coherent,  and  knowledgeable  analysts,  having  access  to  the  same  evidenee,  reach  entirely 
different  conclusions;  how  can  this  be  so?”  The  author  then  proposes  an  answer  that 
suggests  the  fundamental  importance  of  cognitive  process  and  reliance  on 
mental  modeling: 

...  as  they  begin  to  work  on  a  given  problem,  they  do  so  with  different 
perspectives  or  frames  of  reference.  Thus,  their  initial  premises,  hypotheses,  or 
expectations  may  differ.  Since  this  is  so,  they  may  easily  differ  in  judgements 
about  the  relevance  and  weight  of  the  evidence  they  both  observe.  Establishing 
the  weight  or  force  of  evidence  is  a  process  that  relies  on  the  formation  of 
analogies  based  on  experience  and  precedent.  Each  of  the  analysts  will  have  a 
different  ‘matrix’  of  experience  to  draw  upon  in  this  process.  One  analyst 
believes  the  weight  of  a  certain  evidence  item,  on  a  given  hypothesis  they  both 
entertain,  to  be  stronger  than  does  the  other  analyst.  In  combining  or  aggregating 
the  evidence,  one  analyst  assumes  that  the  evidence  items  are  independent  in  their 
impact  on  the  hypothesis;  the  other  analyst  sees  an  interesting  and  subtle  pattern 
of  non-independence  among  the  evidence  items.^^ 

Working  with  “different  perspectives  or  frames  of  reference”  and  relying  on  the 
“formation  of  analogies”  to  formulate  their  analysis  are  unequivocal  indicators  of  high- 
level  cognitive  processes  at  work.  For  the  analysts  to  “see  patterns  of  non¬ 
independence,”  visualization  and/or  mental  modeling  occurs.  Such  observations  suggest 
the  recognition  that  cognitive  models  are  clearly  involved,  even  if  not  clearly  understood. 


F.J.  Hughes,  The  Art  and  Science,  86-87.  Discovery  is  the  process  where  evidence  is  collected 
and  hypotheses  are  generated  and  linked  through  arguments.  Proo/involves  justifying  an  argument 
through  the  process  of  logical  analysis.  Choice  is  the  process  of  deliberating  about  the  arguments  and 
making  decisions  with  respect  to  their  correctness  or  desirability. 

Schum,  17. 

Schum,  17. 
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One  particularly  applicable  DoD  study  proposed  just  such  a  model  as  early  as 
1979.  Targeted  specifically  to  the  domain  of  the  intelligence  analyst,  an  examination  of 
this  important  research  follows. 

A  Relevant  Cognitive  Model.  Cognitive  Processes  in  Intelligence  Analysis:  A 

Descriptive  Model  and  Review  of  the  Literature^'^  published  the  findings  of  a  project 

that  sought  to  “develop  a  cognitive  model  that  would  provide  a  framework  for  the 

description  of  the  mental  processes  used  in  intelligence  analysis.”^^  The  researchers 

contended  that  ‘‘while  based  ultimately  on  data,  intelligence  is  actually  created  by  the 

analyst.  In  other  words,  ‘truth’  is  seldom  hidden  in  the  data;  it  is  constructed  by  the 

analyst.”^^  This  comment  seems  particularly  striking,  both  for  the  period  in  which  it 

was  written,  and  for  its  applicability  to  the  current  environment. 

In  constructing  their  model,  “It  was  assumed  that  a  set  of  common  analytic  task 

processes  exists  that  crosscuts  the  various  intelligence  disciplines  such  as  SIGINT, 

IMINT,  and  HUMINT.”^^  The  conclusion  seems  decidedly  “Heuerarian”^®: 

The  most  telling  result  of  the  research  is  the  clear  implication  that  intelligence 
analysis  is  conceptually  driven  as  opposed  to  data  driven.  What  is  critical  is  not 
just  the  data  collected,  but  also  what  is  added  to  those  data  in  interpreting  them 
via  conceptual  models  in  the  analyst’s  store  of  knowledge.  The  core  junctions  of 


Robert  V.  Katter  and  others,  Cognitive  Processes  in  Intelligence  Analysis:  A  Descriptive  Model 
and  Review  of  the  Literature,  Technical  Report  445  (Alexandria, VA:  U.S.  Army  Research  Institute  for  the 
Behavioral  and  Social  Sciences,  December  1979). 

Katter,  1-1. 

Katter,  ix. 

”  Katter,  1-2. 

Note  previous  reference  to  Heuer’s  use  of  the  concept-driven/data-driven  phraseology. 
Although  Heuer  is  not  cited  in  the  Katter  report,  Heuer’s  use  of  those  precise  terms  in  papers  published  in 
Spring  1979  would  seem  to  precede  that  of  Katter,  if  only  slightly. 
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intelligence  analysis  involve  the  use  of  complex  conceptual  models.  The  ability  to 
use  such  models  is  dependent  upon  individual  capabilities  as  well  as 
environmental  or  work  setting  variables. [italics  mine] 

Chapter  2  of  the  Katter  study  describes  their  COMPARE/CONSTRUCT  model, 

with  refinements  and  relevant  literature  discussed  in  the  document’s  Chapter  3. 

In  summarizing  their  cognitive  model,  the  researchers  concluded: 

[that]  memory  contents  substantially  determine  the  individual’s  automatic 
responses  to,  as  well  as  [their]  aware  experience  of,  new  information.  . . .  Since 
memory  contents  provide  a  large  portion  of  the  information  used  in  making  many 
intelligence  analysis  interpretations  and  estimates,  the  information  contents 
modification  cycle  is  an  important  concept  for  suggesting  ways  to  improve 
intelligence  analysis.^*’ 

Keep  in  mind  that  such  findings  were  derived  in  1979,  well  before  the  arrival  of 
the  ubiquitous  PC  on  every  desktop,  complete  with  INTELINK  and  various  internet- 
based  search  engines.  The  author  submits  that  the  former  analyst-reliance  on  “memory” 
as  cited  above,  has  been  off-loaded  to  databases  and  online  search  results.  The  analyst 
today  relies  far  less  on  human  memory  than  on  the  “collective  memory”  residing  online. 
This  discussion  of  Katter’ s  study  is  provided  to  highlight  two  important  points: 

a.  The  crucial  role  of  cognitive  process  as  a  fundamental  component  of 
intelligence  analysis  has  been  recognized  for  over  two  decades. 

b.  Such  studies,  rather  than  comprising  elements  of  a  core  literature  that 
informs  system  design  for  automated  tools  to  support  intelligence  analysts,  remain 
obscure  and  are  not  integrated  into  a  coherent  design  methodology. 

An  applicable  visualization  reference  model,  accepted  by  NATO  researchers,  is 
examined  below  in  the  sub-section  entitled  European/NATO  Perspectives.  This  tool. 


Katter,  1-2, 
“  Katter,  2-16. 
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called  the  IST-005  Reference  Model,  is  based  on  a  sophisticated  understanding  of 
cognitive  process  and  how  it  relates  to  the  understanding  of  the  entire  dataspace.^^ 

This  research  presented  in  the  NATO  report  clearly  demonstrates  their  commitment  to 
incorporate  visualization  technologies  based  firmly  on  cognitive  design  concepts. 

Applicable  Concepts  of  Psychology  and  Cognitive  Science 

One  would  be  hard-pressed  to  find  an  area  of  study  comprised  of  more  divergent 
and  controversial  views  than  is  found  within  discussions  of  psychology  and  cognitive 
science.  Certainly,  there  is  even  a  lack  of  consensus  in  what  constitutes  “cognitive 
science”  in  general,  with  various  definitions  encompassing  aspects  of  cognitive 
psychology,  philosophy,  neuroscience,  artificial  intelligence  research  and  more. 

This  thesis,  however,  will  seek  to  limit  time  spent  in  such  maelstroms  of 
controversy,  and  instead  will  present  a  subset  of  terms  and  concepts  that  have  a  relatively 
widespread  acceptance.  These  concepts  can  serve  as  important  criteria  to  evaluate  the 
effectiveness  of  the  information  displays  under  discussion.  This  is  by  no  means  an 
exhaustive  list,  but  is  intended  to  demonstrate  the  need  for  incorporating  such 
considerations  in  the  design  phase  of  analytical  tools. 

Cognitive  Ergonomics.  In  an  effort  to  move  theoretical  discussion  into  practical 
application,  the  field  of  cognitive  ergonomics  “analyzes  work  in  terms  of  cognitive 
representations  and  processes,  and  contributes  to  designing  workplaces  that  elicit  and 

NATO,  Visualisation  Study,  4.  The  aspect  of  the  world  that  the  human  wants  to  understand  and 
influence  is  represented  inside  the  computer  as  a  "Dataspace"  accessible  by  computer  processes  ("Engines") 
that  present  their  results  through  displays  to  the  human’s  sensors  (eyes,  ears,  touch...).  From  these  displays, 
the  human  visualises  the  content  of  the  dataspace,  or  rather,  the  aspect  of  the  world  the  dataspace 
represents,  and  is  able  thereby  to  act  effectively. 
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support  reliable,  effective,  and  satisfactory  cognitive  processing.”  Reflecting  the 
interdisciplinary  theme  that  recurs  repeatedly  in  the  literature,  cognitive  ergonomics 
“overlaps  with  related  disciplines  such  as  human  factors,  applied  psychology, 
organizational  studies,  and  human-computer  interaction.”^^ 

It  is  troubling,  however,  to  note  that  a  limited  number  of  cognitive  ergonomics 
research  studies  within  the  domain  of  intelligence  analysis  appear  in  a  review  of  the 
literature.  Notable  exceptions  include  the  previously  cited  Katter  study^'*,  research 
work  originating  from  the  Ohio  State  University  Cognitive  Systems  Engineering 
Laboratory  (CSEL)^^,  papers  authored  by  Roth^^,  the  cited  NATO  Visualization  of 
Massive  Military  Datasets  Study,  and  various  ARDA  initiatives. 


Congruency 

The  notion  of  congruency  encompasses  a  range  of  overlapping 
concepts  that  are  essential  to  ensuring  and  evaluating  the  suitability  of  an  information 
display.  Sometimes  referred  to  as  conformity,  the  concept  “defines  the  degree  of 

MIT  Encyclopedia  of  the  Cognitive  Sciences  (MITECS),  online  edition,  under  the  term 
“Cognitive  Ergonomics.”  URL:<  http://cognet.mit.edu/MITECS/Entry/cara>.  Accessed  26  June  2003. 

MITECS,  “Cognitive  Ergonomics.” 

Katter,  I . 

^  Emily  S.  Patterson  and  others.  Aiding  The  Intelligence  Analyst  In  Situations  Of  Data  Overload: 
A  Simulation  Study  of  Computer-Supported  Inferential  Analysis  Under  Data  Overload,  Monograph,  Report 
ERGO-CSEL-99-02,  Ohio  State  University,  Institute  for  Ergonomics  (Columbus,  OH:  Cognitive  Systems 
Engineering  Laboratory,  1999).  Key  researchers  include  Patterson,  Emilie  M,  Roth,  and  David  D,  Woods. 

^  Emilie  M.  Roth  and  others.  Cognitive  Engineering. 

NATO,  Visualisation  Study. 

The  Advanced  Research  and  Development  Activity  (ARDA)  discussed  in  Chapter  1  sponsors  a 
host  of  applicable  research  projects,  some  of  which  remain  proprietary  or  unpublished. 
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correspondence  between  the  mental  model  in  the  user’s  mind  and  the  actual  system 

presentation.”^^  Stated  in  more  familiar  terms,  it  requires  that  “displays  must  match 

not  only  the  user’s  purposes,  but  also  the  user’s  sensory  and  cognitive  abilities.”^'^ 

Several  conceptual  synonyms,  with  varying  shades  of  meaning,  are  discussed 

throughout  the  literature,  and  can  include  “suitability  for  purpose,”  “fitness  of  purpose,” 

and  “cognitive  fit.”  In  some  cases,  the  implications  of  cognitive  dissonance,  as  advanced 

by  Festinger^^  are  included  as  part  of  the  discussion. 

An  elaboration  upon  the  idea  of  “cognitive  fit”  can  assist  the  understanding  of 

how  it  influences  the  suitability  of  a  design: 

Cognitive  Fit  Theory  states  that  a  solution  to  a  problem  is  ‘an  outcome 
of  the  relationship  between  the  problem  representation  and  problem  solving 
tasks.’  [Vessey,  1991].  The  better  the  ‘fit’  is  between  these  two  constructs,  the 
more  effective  and  efficient  the  problem  solving  process.  Therefore,  when 
developing  information  visualizations,  the  developer  must  pay  attention  to  the 
tasks  performed  by  the  decision-maker  if  the  visualization  is  to  be  successful. 

Although  termed  in  various  ways,  a  critical  assessment  of  “suitability”  remains 

central.  Bremer  advances  the  notion  of  verisimilitude^^  to  assess  the  validity  of  a 

model.  “A  model  has  verisimilitude  to  the  degree  that  its  behavior  matches  the  behavior 


NATO,  Visualisation  Study,  59. 

NATO,  Visualisation  Study,  iv. 

Leon  Festinger,  A  Theory  of  Cognitive  Dissonance  (Stanford,  CA:  Stanford  University  Press, 
1957):  12-13.  Festinger  states  that  “two  elements  are  dissonant  if,  for  one  reason  or  another,  they  do  not  fit 
together.  They  may  be  inconsistent  or  contradictory.” 

David  P.  Tegarden,  “Business  Information  Visualization,”  Communications  of  the  Association 
For  Information  Systems  1,  Paper  4,  electronic  journal,  January  1999:  11,  URL:<http://cais.isworId,org>, 
under  the  keyword  “Tegarden,”  accessed  10  January  2003. 

Verisimilitude  as  defined  by  Webster  is  the  appearance  or  semblance  of  truth;  likelihood;  or 

probability. 


34 


of  its  referent  system,  and,  other  things  being  equal,  the  greater  a  model’s  verisimilitude, 
the  more  confidence  we  place  in  it.”’’’^ 

For  the  purpose  of  this  thesis,  a  generalized  definition  will  suffice  to  make  the 
case  for  the  concept’s  fundamental  importance.  Plainly  stated,  congruency  should  serve 
as  a  basic  criteria  to  assess  whether  a  dh^XdLy  facilitates  or  inhibits  the  task  it  is  claimed 
to  support.  There  is  scant  evidence  to  suggest  that  such  criteria  are  considered  as  design 
factors  or  evaluative  measures  in  the  current  paradigm. 

Domain  Specificity  /  Domain  Independence 

Several  fields  of  study,  including  cognitive  science,  recognize  the  concepts  of 
domain  specificity  and  domain  independence.  Fundamentally,  it  is  agreed  that  “cognitive 
abilities  are  domain-specific  to  the  extent  that  the  mode  of  reasoning,  structure  of 
knowledge,  and  mechanisms  for  acquiring  knowledge  differ  in  important  ways  across 
distinct  content  areas.”^^  In  practical  terms  this  means  that  the  ways  in  which  knowledge 
is  acquired,  understood,  and  expressed  is  unique  to  that  domain,  while  domain 
independence  conversely  implies  a  model  that  is  generally  applicable  across  many 
domains. 

Such  principles  are  routinely  used  to  inform  design  considerations.  Within  the 
intelligence  domain,  word  processing  software  exhibits  domain  independence,  whereas 
electronic  light-table  (ELT)  software  for  the  imagery  analyst  exhibits  domain  specificity. 


Stuart  A.  Bremer,  “Evaluating  GLOBUS,”  in  The  GLOBUS  Model:  Computer  Simulation  of 
Worldwide  Political  and  Economic  Developments,  ed.  Stuart  A.  Bremer  (Boulder,  CO:  Westview  Press, 
1987):  724. 

MITECS,  “Domain  Specificity.” 
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While  both  concepts  have  appropriate  applications,  either  decision  has  far- 
reaching  implications  for  the  end-user.  One  criticism  of  a  domain  specific  design  is  that 
“its  development  effort  cannot  be  amortized  over  several  different  systems.”^^ 

A  domain  independent  approach,  however,  ensures  the  benefits  of  “economy  of  scale,” 
as  evidenced  by  most  mass-marketed  products,  including  the  personal  computer  and  its 
suite  of  business-based  software.  In  the  case  of  what  is  termed  in  this  study  as  the  PC 
Paradigm,  domain  independence  is  manifested  as  a  generic,  one-size-fits-all  design 
optimized  to  maximize  market  share.  While  commercially  impressive,  such  an  approach 
fails  to  address  the  needs  of  those  who  require  a  domain-specific  design. 

Visualization  and  Mental  Models 

Even  though  we  navigate  daily  through  a  perceptual  world  of  three  spatial 
dimensions  and  reason  occasionally  about  higher  dimensional  arenas  with  mathematical 
ease,  the  world  portrayed  on  our  information  displays  is  caught  up  in  the  two- 
dimensionality  of  the  endless  flatlands  of  paper  and  video  screen. 

Edward  R.  Tufte,  Envisioning  Information 

The  above  quotation  by  Edward  Tufte  cogently  captures  the  essence  of  a  problem 
as  central  to  the  intelligence  community  as  it  is  to  graphics  designers.  The  problems 
intelligence  analysts  face  are  rarely  reducible  to  the  simplistic  “row  and  column/either-or” 
solutions  often  supported  by  spreadsheet  and  matrix-based  tools.  Instead,  analysis 
examines  concepts,  relationships,  and  multivariate  data  by  forming  mental  representations 
or  visualizations  of  the  problem. 

It  is  important  to  emphasize  that  the  term  visualization  in  this  context  refers  to  a 
process,  internalized  within  the  user,  and  only  in  some  cases  expressed  externally  through 
a  graphic,  illustration,  or  computer  display.  The  tendency  to  equate  visualization  to  a 

Harold  Thimbleby,  User  Interface  Design  (New  York:  ACM  Press,  1990):  213. 
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visual  display  underscores  the  tool-centric  paradigm  in  place  and  fosters  the  notion  of 
visualization  as  a  product  or  “deliverable”  achieved  through  state-of-the-art  information 
displays.  “Visualization  is  not  a  data  display,  however  ingenious.  It  is  one  route  to 
understanding,  another  route  being  logical  analysis.”^^  Recognizing  this  key  distinction, 

TO 

NATO  researchers  have  incorporated  the  concept  into  established  design  guidance, 

referred  to  as  the  IST-05  Reference  Model 

The  IST-05  Reference  Model  emphasizes  that  "Visualisation"  does  not  refer  to 
displays  on  computer  screens,  no  matter  how  evocative  and  dramatic  they  may  be. 
Screen  displays  are  important  to  the  visualisation  process,  in  that  a  good  display, 
by  promoting  a  useful  visualisation  of  the  data  being  understood,  provides  a 
natural  link  between  the  human’s  understanding  and  those  data.  Engines  and  I/O 
devices  are  essential  aspects  of  the  visualisation  support,  and  indeed  are  the  only 
parts  of  the  Reference  Model  subject  to  engineering  design  and  modification.  To 
design  useful  engines  and  devices,  however,  it  is  necessary  that  the  designer 
understand  the  human  process  of  visualisation. 

The  concept  of  visualization-as-cognitive-process  seems  to  be  broadly  understood 
in  the  design  community.  Recurring  themes  across  multiple  disciplines  are  readily 
apparent  and  such  convergences  should  indicate  where  design  standardization  and 
specification  could  begin  on  common  ground.  It  is  much  less  apparent  that  such  design 
fundamentals  are  incorporated  in  currently-available  tools  for  the  analyst. 

Elements  of  graphic  design  pertaining  to  this  discussion  are  examined  in  the 
following  section.  Such  explorations  illustrate  that  the  important  link  between  cognitive 


NATO,  Visualisation  Study,  iii-iv. 

Presented  and  sponsored  by  the  members  of  NATO  Research  Study  Group  IST-013/RTG-002 
for  the  RTO  (Research  and  Technology  Organization)  Information  Systems  Technology  Panel  (1ST).  The 
final  report  in  its  entirety  is  cited  as  NATO,  Visualisation  Study. 

NATO,  Visualisation  Study,  4. 


37 


process  and  information  representation  is  appreciated  even  in  non-systems-oriented  fields 

80 

of  endeavor.  As  noted  in  Envisioning  Information^ 

Escaping  [two-dimensional]  flatland  and  enriching  the  density  of  data  displays  are 
the  essential  tasks  of  information  design.  ...  By  giving  the  focus  over  to  data 
rather  than  data-containers,  these  design  strategies  are  transparent  and  self- 
effacing  in  character. 

The  characterization  of  current  information  displays  as  “data  containers”  seems 
particularly  apt.  Commitment  to  fundamental  design  principles  that  “transparently 
facilitate”  the  cognitive  processes  involved  in  visualization  must  be  incorporated  into  the 
description  of  design  requirements  for  meaningful  analytic  tools. 


Tenets  of  Graphic  Design 

Fundamental  principles  of  composition,  color,  balance,  and  line  guide  the  pursuit 
of  excellence  in  visual  representations  of  all  types.  From  the  fine  arts,  to  photography,  to 
advertising,  widely-accepted  elements  of  design  are  utilized  to  most  effectively  “get  the 
message  across.”  While  acknowledging  the  subjective  nature  of  what  constitutes  “good 
design”  within  the  arts,  a  school  of  thought  has  arisen  surrounding  the  visual  display  of 
information  as  well.  This  subset  of  “information  display  as  tooV’  is  the  focus  of  this 
study  and  is  of  central  importance  to  the  intelligence  analyst.  Fundamental  design 
principles  are  available  to  guide  the  design  of  such  tools,  and  the  question  remains,  “Are 
the  current  information  display  technologies  available  to  the  analyst  incorporating  the 
optimal  design  elements?” 


Edward  R.  Tufte,  Envisioning  Information  (Cheshire,  CT:  Graphics  Press,  1990):  33. 
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Among  the  most-respected  proponents  of  excellence  in  information  display  is 
Edward  R.  Tufte  whose  works  The  Visual  Display  of  Quantitative  Information 

8 1 

and  Envisoning  Information  are  considered  classics  in  the  field.  Tufte  suggests: 

Excellence  in  . . .  graphics  consists  of  complex  ideas  communicated  with  clarity, 
precision,  and  efficiency.  Graphical  displays  should: 

•  Show  the  data. 

•  Induce  the  viewer  to  think  about  the  substance  rather  than  about  the 
methodology,  graphic  design,  the  technology  of  graphic  production,  or 
something  else. 

•  Avoid  distorting  what  the  data  have  to  say. 

•  Make  large  data  sets  coherent. 

•  Encourage  the  eye  to  compare  different  pieces  of  data. 

•  Reveal  the  data  at  several  layers  of  detail,  from  a  broad  overview  to  the 
fine  structure. 

•  [emphasizing  that]  ...graphics  reveal  the  data. 

Tufte  would  likely  question  whether  the  familiar  refrain  of  information  overload 
is  actually  the  problem  or  merely  symptomatic.  To  illustrate  (simultaneously,  though 
perhaps  unintentionally,  demonstrating  that  the  data  overload  is  not  a  new  phenomenon) 
he  quotes  Richard  Saul  Wurman  as  stating  in  1976  that  “everyone  spoke  of  an 

S'? 

information  overload,  but  what  there  was  in  fact  was  a  non-information  overload." 

Indeed  the  display  of  non-information,  variously  characterized  within  the 
intelligence  community  as  noise,  chajf,  chatter,  or  more  broadly,  distractors,  is  a  vexing 
problem  of  current  representations,  but  not  one  that  should  be  considered  insurmountable. 

Edward  R.  Tufte,  The  Visual  Display  of  Quantitative  Information  (Cheshire,  CT:  Graphics 
Press,  1983):  13. 

Tufte,  Visual  Display,  90. 


39 


Tufte  observes: 


Confusion  and  clutter  are  failures  of  design,  not  attributes  of  information.  And  so 
the  point  is  to  find  design  strategies  that  reveal  detail  and  complexity  -  rather  than 
to  fault  the  data  for  an  excess  of  complication.  Or,  worse,  to  fault  viewers  for  a 
lack  of  understanding.®^ 

Apparently  this  dictum  resonates  as  a  fundamental  truism,  as  Tufte  concluded  in 
his  earlier  text: 

What  is  sought  in  designs  for  the  display  of  information  is  the  clear  portrayal  of 
complexity.  Not  the  complication  of  the  simple;  rather  the  task  of  the  designer  is 
to  give  visual  access  to  the  subtle  and  the  difficult  -  that  is,  the  revelation  of  the 
complex.^'^  {italics  mine) 

Frankel®^  addresses  the  concept  of  easing  the  task  of  complexity  representation 
by  pragmatic  approaches  to  image  selection  and  presentation  in  the  field  of  science. 
Pointing  out  that  modem  research  is  increasingly  interdisciplinary  in  nature,  the  need  for 
communicating  findings  and  concepts  to  those  outside  of  the  immediate  realm  of  the 
researcher’s  expertise  is  cmcial.  An  apt  parallel  can  be  drawn  to  the  modem  intelligence 
analyst  as  they  are  increasingly  called  upon  to  provide  the  results  of  their  analysis  to  a 
diverse  and  ever-widening  audience  of  decision-makers,  both  military  and  civilian. 

Envisioning  Science  provides  technical  guidance  and  specific  applications 
suggestions  for  maximizing  the  information  content  of  images,  including  photography 
and  microscopy  products,  to  diverse  audiences.  Important  implications  for  the 
improvement  of  intelligence-related  visual  products  can  be  inferred.  Cautioning  that 
immediate  understanding  is  not  always  obvious  and  cannot  be  assumed,  Frankel  states 


Tufte,  Envisioning,  53. 

Tufte,  Visual  Display,  191. 

Felice  Frankel,  Envisioning  Science:  The  Design  and  Craft  of  the  Science  Image  (Cambridge, 
MA:  The  MIT  Press,  2002). 
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that  “creating  order  is  the  first  step  in  making  a  successful  image.  ...  You  know  what 

to  look  for  and  what  to  disregard,  but  the  first-time  viewer  sees  everything.”  The  author 

offers  the  following  way  to  assist  the  viewer  (or  the  analyst)  in  making  connections:®^ 

Encouraging  the  viewer  to  compare  the  important  component  as  it  relates  to 
something  else  helps  to  clarify  what  is  essential  and  engages  the  viewer.  . . .  When 
you  include  another  form  in  the  image,  let’s  call  it  B,  the  viewer  is  able  to  make 
one  of  the  following  comparisons  between  components  A  and  B: 

A  looks  like  B 

A  is  different  from  B 

A  has  some  relation  to  B 

A  has  nothing  to  do  with  B 

The  foregoing  example  illustrates  that  finding  relevance  in  complex  datasets  is 
not  a  problem  unique  to  the  intelligence  community.  Solutions  are  being  proposed,  and 
techniques  implemented,  in  other  disciplines  and  fields  of  study  that  may  have  direct 
application  to  similar  problems  facing  the  IC. 

Once  again  the  importance  of  intersecting  paradigms  or  “schools  of  thought”  is 
evident.  The  design  principles  for  optimizing  information  representation  espoused  by 
Tufte,  Frankel  and  others  are  interwoven  with  research  in  the  fields  of  cognitive  science 
and  mental  modeling  processes.  The  desired  emphasis  is  on  the  goal  of  information 
transference  and  subsequent  knowledge  formation  and  not  the  technology  involved.  A 
critical  examination  of  the  current  tools  in  use  is  necessary  to  determine  if  such  design 
principles  are  truly  driving  the  creation  of  new  “analyst-assist  products”  and  the 
improvements  of  existing  tools  and  their  features. 


Frankel,  28-29. 
Frankel.  46-47. 
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Human-Computer  Interface  (HCI)  Design  Considerations 

The  essential  dilemma  of  a  computer  display:  at  every  screen  are  two  powerful 
information-processing  capabilities,  human  and  computer.  Yet  all  communication 
between  the  two  must  pass  through  the  low-resolution,  narrow-band  video  display 
terminal,  which  chokes  off  fast,  precise,  and  complex  communication. 

Edward  R.  Tufte,  Envisioning  Information 

The  “dilemma”  described  above  illustrates  the  challenge  of  creating  an  effective 

bridge  between  human  and  machine,  commonly  referred  to  as  the  human-computer 

interface,  or  more  simply,  HCI.  Although  the  term  “interface”  is  by  no  means  limited  to 

computing,  for  the  purposes  of  this  thesis,  the  following  definition  will  be  accepted: 

An  interface  is  a  describable  structure  through  which  a  user  interacts  with  a 
computer  or  task.  ‘Interface’  is  a  noun  that  describes  structures  or  mechanism, 
whereas  ‘interact’  is  a  verb  that  designates  process.  ‘Interaction’  always  is  done 
through  an  ‘interface’  and  neither  can  be  completely  described  without  reference 
to  the  other.®® 

A  considerable  literature  exists  in  the  field  of  human-computer  interface  design 
and  this  section  is  not  intended  as  a  primer  on  the  subject.  The  discussion  will  however, 
bring  forth  key  design  concerns  and  introduce  basic  cognitive  engineering  objectives.  An 
example  using  a  National  Aeronautics  and  Space  Administration  (NASA)  scenario 
relevant  to  the  intelligence  domain  is  presented. 

Norman  offers  a  straightforward  heuristic  for  evaluating  an  interface:  “The 
usability  of  . . .  human  computer-computer  interfaces  depends  on  how  the  cognitive 
abilities  of  the  user  are  either  facilitated  or  stressed  by  the  interface.”®^ 

Norman’s  pronouncement  brings  important  attention  to  the  human  aspect  of  HCI  design. 


®  NATO,  Visualisation  Study,  55. 
Norman,  2. 


42 


In  many  cases,  the  design  effort  is  placed  on  the  software  engineering  or 
“computer”  side  of  “human-computer  interface”,  while  the  “human”  component  is 
lacking  or  inadequate.  Cognitive  engineering,  discussed  below,  seeks  to  address  this 
imbalance.  Sub-optimal  design  can  occur  when  software-centric  methods  are  exclusively 
employed.  In  such  environments  it  is  possible  that  “designers  who  are  unaware  of  human 
factors  knowledge  about  user  interface  design  do  not  perceive  a  breakdown  if  one  of  the 
design  guidelines  is  violated.”^*’  Roth  and  others  caution  that  “the  objective  is  to 
maximize  the  overall  performance  of  this  joint  human-computer  cognitive  system,  which 
is  not  always  the  same  as  maximizing  the  performance  of  the  software  when  considered 
in  isolation.”^’ 

Design  or  Ad  Hoc  Creation?  There  are  those  who  question  whether  a 
structured  design  methodology  is  evident  in  all  cases.  System  complexity,  escalating 
customer  requirements,  lack  of  configuration  management,  and  frequent  changes  in 
software  maintenance  personnel  can  all  contribute  to  “feature  creep”  and  “design-by¬ 
accumulation.”  Thimbleby  observes: 

At  present  the  best  interactive  systems  are  built  partly  by  inspiration,  possibly 
guided  by  formal  considerations,  but  in  large  part  by  ad  hoc  creation.  Many  such 
accretions  are  details  for  handling  special  cases  and  are  usually  afterthoughts  to 
the  main  thrust  of  design.  ...  Despite  adherence  to  user-engineering  principles,  it  is 
unlikely  that  a  coherent  system  will  result.^^ 


Harold  Reiterer,  “User  Interface  Design  Assistant,”  in  Computers  Ar  Assistants:  A  New 
Generation  of  Support  Systems,  ed.  Peter  Hoschka  (Mahwah.NJ:  Lawrence  Erlbaum  Associates,  Inc., 
1996):  161. 

Roth  and  others.  Cognitive  Engineering,  9. 

^  Thimbleby,  201-202. 
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The  prospect  of  creating  “incoherent  systems”  through  ad  hoc  design  is  sobering, 
but  it  is  not  the  only  potential  pitfall.  A  phenomenon  Thimbleby  refers  to  as  intellectual 
sacrifice  has  been  observed  to  apparently  remove  rationality  from  the  equation  in  some 
cases: 

Using  a  particular  user  interface  causes  people  to  become  committed  to  it.  It 
becomes  ‘right’  and  apparently  better  than  competing  systems  even  if  [the 
competing  systems]  would  have  been  right  had  they  been  used  first.  The  users 
simply  make  a  choice  of  axioms  (and  reasoning  methods),  then  use  them,  then 
become  irrevocably  committed  to  their  original  choice  -  and  it  also  becomes 
almost  impossible  to  disabuse  them  of  ‘incorrect’  aspects  of  the  user  interface 
design.^^ 


A  Cognitive  Engineering  Approach.  The  study  entitled  Cognitive  Engineering: 
issues  in  User-Centered  Design^*  presents  a  concise  explanation  of  how  a  cognitive 
engineering  approach  centers  on  what  the  user  is  trying  to  achieve  (the  task)  and  the 
domain  in  which  it  is  performed  (domain  specificity).  The  following  summarized 
discussion  and  guidelines  are  particular  applicable  to  the  design  environment  that  exists 
within  the  intelligence  community:^^ 

The  guiding  tenet  of  cognitive  engineering  is  that  consideration  of  the  users  and 
the  tasks  they  will  be  performing  with  the  aid  of  a  computer  system  should  be 
central  drivers  for  system  design  specification.  HCI  is  not  to  be  viewed  as 
peripheral  to  the  primary  concern  of  software  engineering.  Instead  a  user 
centered,  or  practice-centered,  system  design  approach  is  embraced  in  which  the 
questions  that  drive  design  include  the  following: 


Thimbleby,  342. 

^  Roth  and  others,  Cognitive  Engineering.  The  authors  define  Cognitive  Engineering  as  an 
interdisciplinary  approach  to  the  development  of  principles,  methods,  tools,  and  techniques  to  guide  the 
design  of  computerized  systems  intended  to  support  human  performance.  . . .  The  basic  unit  of  analysis  and 
design  in  cognitive  engineering  is  a  cognitive  system. 

Roth  and  others.  Cognitive  Engineering,  2-3. 
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•  What  are  the  goals  and  constraints  in  the  application  domain? 

•  What  range  of  tasks  do  domain  practitioners  perform? 

•  What  strategies  do  they  use  to  perform  these  tasks  today? 

•  What  factors  contribute  to  task  complexity? 

•  What  tools  can  be  provided  to  facilitate  the  work  of  domain  practitioners  and 

achieve  their  goals  more  effectively? 

Such  questions  could  serve  as  the  basis  for  evaluative  criteria  when  examining 
any  proposed  or  existing  system  purported  to  augment  intelligence  analysis.  Looking 
closely  at  the  questions  above,  this  cognitive  engineering  approach  directly  addresses 
many  of  the  key  issues  of  concern  for  the  IC. 

“Translating”  the  language  of  cognitive  science  terminology  into  terms  and 
concepts  more  familiar  to  the  analyst  facilitates  such  an  examination.  A  substitution  of 
the  phrase  ''the  intelligence  field"  for  “application  domain,”  and  "intelligence  analysf'  for 
“domain  practitioner”  clarifies  the  connection.  Likewise,  “strategies  used  to  perform 
tasks”  could  become  "analytical  methodologies."’  Factors  that  “contribute  to  task 
complexity”  within  the  IC  are  well  known  and  the  subject  of  considerable  discussion.^^ 
The  intelligence  domain  must  also  contend  with  an  environment  characterized  by  massive 
data^  further  indicating  the  need  for  a  comprehensive  design  effort. 


^  Heuer.  In  Psychology  of  Intelligence  Analysis,  Heuer  discusses  numerous  factors  that  contribute 
to  the  complexities  of  intelligence  analysis  including  perceptual  challenges,  memory  constraints,  satisficing 
and  premature  closure,  cognitive  biases,  information  uncertainty  and  many  others. 

Hughes,  The  Art  and  Science,  8 1 .  Hughes  defines  massive  data  as  having  multiple  dimensions 
that  may  cause  difficulty,  some  of  which  include  volume  or  depth,  heterogeneity  or  breadth,  and 
complexity. 
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Cognitive  Task  Analysis  (CTA)  is  one  methodology  that  can  be  employed  as  part 
of  such  an  approach  to  comprehensive  cognitive  engineering  design,  and  will  be 
introduced  in  a  separate  sub-section  below. 

HCl  and  Appropriate  Metaphor.  Any  discussion  of  human-computer  interface 
issues  is  closely  linked  to  the  concept  of  employing  the  appropriate  user  interface 
metaphor  as  discussed  in  Chapter  1.  In  the  current  PC  Paradigm  environment,  the 
desktop  metaphor  continues  to  remain  dominant  for  reasons  that  likely  include  some 
degree  of  intellectual  sacrifice  or  “interface  anchoring”  as  outlined  above,  as  well  as 
commercial  market  influence.  Considerable  evidence  can  be  found,  however,  to  suggest 
its  suitability  is  in  question  throughout  many  information  domains.  Remarks  from  the 
civil  sector  include  the  observation  that  “while  the  desktop  metaphor  has  become  a  very 
common  one  on  some  computer  systems,  it  may  be  inappropriate  for  the  town  planner, 
utility  manager  or  ecologist.”^® 

Reasons  for  such  “inappropriateness”  may  be  grounded  in  the  domain-general 

approach  of  the  current  paradigm  in  trying  to  be  all  things  to  all  users.  Appreciation  for 

the  fact  that  designers  can  consider  a  range  of  potential  metaphors  is  not  new,  however, 

as  Fundamentals  of  Interactive  Computer  Graphics  pointed  out  in  1982: 

It  may  well  be  desirable  that  the  conceptual  model  be  similar  to  the  concepts  with 
which  the  user  is  already  familiar.  On  the  other  hand,  some  other  model  may  be 
more  powerful  and  efficient.  The  more  adaptable  the  user  community  is  to  new 
ideas,  the  more  the  designer  can  opt  for  power  over  familiarity.^^  (italics  mine) 


Medyckyj -Scott,  205. 

J.D,  Foley  and  A,  Van  Dam,  Fundamentals  of  Interactive  Computer  Graphics  (Reading,  MA: 
Addison-Wesley  Publishing  Co.,  1982):  240. 
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A  NASA  Analogy.  A  comparison  with  the  problems  NASA  has  faced  in 
planetary  exploration  provides  a  relevant  analogy.  Referring  to  the  activities  or  tasks  that 
are  performed  in  this  domain  as  exploration  behaviors,  McGreevy  suggests  “distilling” 
these  behaviors  and  applying  them  to  the  design  of  user  interfaces  for  exploration 
systems.'*^ 

As  McGreevy  points  out,  “this  kind  of  distillation  originally  informed  the  desktop 
metaphor  as  well,  but  the  behaviors  of  interest  in  that  case  were  ‘desktop,’  paper-work- 
oriented  behaviors. The  desktop  metaphor  is  inappropriate,  however,  for  the  tasks 
and  behaviors  associated  with  planetary  exploration.  McGreevy’ s  concern  is  with 
attempting  to  achieve  a  “spatial  linkage  with  the  environment  being  explored,”  and 
observes  that  “keystroke-oriented  interactions,  and  point- and-c lick  interactions  with  icons 
representing  ...  documents,  files,  [and]  paperwork  tools”  are  quite  inadequate. 

The  dissonant  condition  described  in  McGreevy’s  example  is  a  compelling 
example  of  the  non-congruency  between  the  functionality  of  the  tool  provided  and  the 
needs  of  the  user  working  within  a  specific  domain  -  a  fundamental  critique  of  the 
existing  paradigm  as  discussed  in  this  thesis. 

European/NATO  Perspective 

Both  the  European  Union  and  NATO  are  concerned  with  the  consequences  of 
rapid  technology  infusion.  The  following  section  summarizes  key  findings  of 
particularly  applicable  documents. 

M.W.  McGreevy,  “Virtual  Reality  and  Planetary  Exploration,”  in  Virtual  Reality:  Applications 
and  Explorations,  ed.  Alan  Wexelblat  (  Cambridge,  MA;  Academic  Press  Professional,  1993);  181. 

McGreevy,  181. 

McGreevy,  182. 
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Relevant  Views  of  the  EU.  Offering  a  pan-European  perspective,  the  final  report 


of  the  Advanced  Science  and  Technology  Policy  Planning  Network  (ASTPP)  was 
reviewed.'”^ 

The  report  noted  a  key  challenge:  “The  production  of  highly  sophisticated 

products  makes  increased  demands  on  the  science  base,  necessitating  inter  and  trans- 

disciplinary  research  and  the  fusion  of  heterogeneous  technological  trajectories.”'*^ 

Concern  with  such  “trajectories”  is  well-placed  as  fragmented  research  efforts 

result  in  an  uncoordinated  sprawl  of  design  solutions,  each  addressing  only  particular 

aspects  of  the  larger  problem.  Recognizing  the  key  influence  of  market  forces  in  this 

process,  the  panel  advocates  a  “variable  geometry  approach”  to  ensure  flexibility. 

Such  flexibility  can  help  to  avoid  being  swept  before  the  “technological 

tide”  and  assist  in  mitigating  the  effects  of  what  the  report  identifies  as  the  anticipation 

and  control  dilemma^^,  articulated  as  follows: 

At  an  early  stage  of  technology  development,  the  nature  of  the  technology  (and 
the  articulation  of  interests)  are  still  malleable  -  but  it  is  unclear  what  the  effects 
and  impacts  will  be.  By  the  time  these  become  clear,  the  technology  is 
entrenched  and  vested  interests  make  it  difficult  to  change  the  technology.'*'^ 

Similiar  observations  appear  to  be  a  common  lament.  As  noted  by  Davis  and 

Medyckyj -Scott  “introducing  changes  to  the  system,  to  improve  interaction  [after  the 


Kuhlmann  and  others.  Contributors  to  the  report  included  members  from  Germany,  France, 
Greece,  Finland,  Great  Britain,  Portugal,  Austria,  The  Netherlands,  and  Spain. 

Kuhlmann  and  others,  7. 

Kuhlmann  and  others,  15.  “Policy  has  evolved  from  ‘science  policy,’  to  ‘science  and 
technology  policy’  and  latterly  to  ‘innovation  policy,’  with  a  corresponding  shift  in  policy  emphasis  from 
‘science  push’  to  ‘market  pull’  and  latterly  to  a  realisation  that  ‘variable  geometry’  policy  mixes  are  needed 
to  ‘manage’  complex  ‘innovation  systems.’” 

Kuhlmann,  42. 

Kuhlmann,  42. 
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system  is  designed  and  built]  can  be  very  difficult  due  to  technical  and  organisational 
inertia.”^*’®  The  ASTPP  report  underscores  the  pervasive  nature  of  market  influence 
and  observes  that  “the  recent  economics  and  sociology  of  technology  have  traced  the 
increasing  path  dependencies  in  technological  development.”'*^^  Restated,  market  and 
social  forces  are  the  drivers  for  technological  development.  Fitness-for  purpose”  design 
considerations  are,  not  surprisingly,  absent. 

Unguided  “technological  trajectories”  also  create  a  “co-production  of  impacts,” 
that  even  if  unanticipated,  may  continue  to  exert  influence  far  beyond  the  initial 
introduction  of  the  product  or  feature.  The  study  presents  the  “QWERTY  keyboard  of 
typewriters,  but  now  also  of  computers,  [as]  a  well-known  example.”"** 


NATO  Perspectives.  The  North  Atlantic  Treaty  Organization  (NATO)  relies  on 
its  Research  and  Technology  Organization  (RTO)  for  defense-related  research  and 
technology  assessment.'" 


C.  Davis  and  D.  Medyckyj -Scott,  “Introduction;  The  Importance  of  Human  Factors,”  in 
Visualization  in  Geographic  Information  Systems,  ed.  Hilary  M.  Hearnshaw  and  David  J.  Unwin  (  New 
York:  John  Wiley  and  Sons,  1994):  191. 

Kuhlmann  and  others,  42. 

Kuhlmann  and  others,  42.  The  continued  use  of  the  familiar  QWERTY  keyboard  layout  is 
often  used  to  demonstrate  the  “staying  power”  of  an  accepted  paradigm  long  after  its  usefulness  has 
expired.  Although  allegedly  intended  to  slow  the  manual  data-entry  rate  of  early  mechanical  systems  to 
avoid  jammed  keys,  the  veracity  of  the  story  is  undetermined.  In  any  event,  while  more  efficient  keyboard 
layouts  are  available,  the  original  QWERTY  design  served  to  transition  the  dominant  paradigm  from  the 
manual  typewriter  to  the  computer  keyboard,  where  it  remains  dominant  today. 

’  *  ’  NATO,  Visualisation  Study,  ii.  RTO  is  the  single  focus  in  NATO  for  Defence  Research  and 
Technology  activities.  Its  mission  is  to  conduct  and  promote  cooperative  research  and  information  exchange. 
The  objective  is  to  support  the  development  and  effective  use  of  national  defence  research  and  technology  and  to 
meet  the  military  needs  of  the  Alliance,  to  maintain  a  technological  lead,  and  to  provide  advice  to  NATO  and 
national  decision  makers.  ...  It  comprises  a  Research  and  Technology  Board  (RTB)  as  the  highest  level  of 
national  representation  and  the  Research  and  Technology  Agency  (RTA),  a  dedicated  staff  with  its  headquarters 
in  Neuilly,  near  Paris,  France. 
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Of  particular  note  for  the  purposes  of  this  thesis  are  three  NATO  RTO  reports 
concerning  visualization  of  massive  data  sets^'^,  commercial-off-the-shelf  (COTS) 
integration  issues^'^,  and  perspectives  on  human  factors  in  the  21*'^  century."'*  These 
comprehensive  studies  offer  a  wealth  of  relevant  information  and  a  complete  review  of 
each  document  is  beyond  the  scope  of  this  study.  Excerpted  highlights  of  particularly 
relevant  concepts  are  provided  below. 

The  NATO  Research  Study  Group  “IST-013/RTG-002”  has  accepted  a 
sophisticated  visualization  design  model  that  incorporates  cognitive  process 
considerations,  called  the  IST-005  Reference  Model. The  model  “emphasizes  that 
‘visualisation’  does  not  refer  to  displays  on  computer  screens,  no  matter  how  evocative 
and  dramatic  they  may  be,”  and  instead  seeks  to  match  displays  to  human  sensory 
capabilities."^  Discussions  of  the  IFIP  (International  Federation  for  Information 
Processing)  model*"  and  Layered  Protocol  Theory*’®  are  presented. 

A  recent  (2000)  taxonomy  for  describing  visualisation  systems  called  RM-Vis  is 
discussed.’*^  Described  as  a  “framework  for  the  development  of  visualisation  reference 
models  that  focuses  on  the  application  of  visualisation  solutions  within  particular  domain 

NATO,  Visualisation  Study. 

“^NATO,  COTS  Study. 

North  Atlantic  Treaty  Organization,  Human  Factors  in  the  IF’  Century,  RTO-MP-77, 
AC/323(HFM-062)TP/38,  (Neuilly-Sur-Seine  Cedex,  France:  Research  and  Technology  Organization,  May 
2002). 

NATO,  Visualisation  Study,  iv, 

NATO,  Visualization  Study,  4. 

NATO,  Visualisation  Study,  58. 

NATO  Visualisation  Study,  61-64. 

NATO  Visualisation  Study,  126. 
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contexts,”  the  model  “classifies  visualisation  applications  in  a  three-dimensional  space, 
with  the  dimensions  of  domain  context,  visualisation  approach,  and  descriptive  aspects." 

Note  that  such  tools  for  generating  domain- specific  reference  models  to  build 
visualization  systems  are  available  and  in  use  within  the  NATO  research  community. 
While  not  the  only  such  effort  (or  model),  the  point  to  be  made  is  that  in  the  NATO 
examples  cited,  design  initiative  is  being  actively  pursued  rather  than  passive 
aceeptance  of  commercially  available  solutions.  An  additional  concern  arises  in 
questioning  whether  sufficient  coordination  is  being  made  between  the  NATO  and  U.S. 
efforts  in  information  display  development.  This  is  an  issue  that  should  concern  “fitness- 
of-purpose”  assessments  for  personal  computer  systems  as  well  as  specialized,  high-end 
workstations. 

Illustrating  the  fact  that  NATO  recognizes  the  problem  of  “massive  data”  and 
intends  to  actively  pursue  visualization  technologies  as  a  solution,  the  RTO  offers  the 
following  (excerpted)  recommendations 

Section  10.4.1  General  recommendations; 

Accelerate  the  development  and  deployment  of  information  visualisation 
throughout  NATO  eountries  and  PfP  [Partnership  for  Peace]  by  promoting 
appropriate  use  of  visualisation  for  improved  information  accessibility, 
operational,  filtration,  extraction  and  understanding.  . . . 

Stress  the  importance  of  evaluative  testing  as  opposed  to  subjective  "beauty 
contests"  in  determining  the  effectiveness  of  visualisation  techniques. 

The  fundamental  importance  of  congruency  or  “cognitive  fit”  as  emphasized  in 

this  thesis  is  also  apparent  in  the  NATO  design  considerations,  noting  that  “displays  must 

match  not  only  the  user’s  purposes,  but  also  the  user’s  sensory  and  cognitive  abilities. 

NATO,  Visualisation  Study,  134. 

NATO,  Visualisation  Study,  iv. 
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Recommendations  for  developers  include  the  following  guidance: 

Display  requirements  are  different  for  analysis  and  for  visualisation.  Analysis  is 
eased  by  an  uncluttered  display  that  allows  the  focal  objects  to  stand  out  clearly 
and  that  illustrates  their  relationships,  whereas  visualisation  generally  requires 
copious  context,  possibly  with  the  focal  elements  highlighted  in  some  way. 

In  marked  contrast,  HCI  design  guidance  offered  in  key  U.S.  Department  of 

Defense  Intelligence  Information  Program  (DoDIIS)  documents,  reviewed  below, 

suggests  a  strong  COTS-product-driven  influence  rather  than  domain-specific  design. 

Department  of  Defense  (DoD)  Guidance 

Conceptual  design  and  idealism  aside,  a  pragmatic  examination  of  the  present 
reality  of  PC-based  systems  employment  within  the  Department  of  Defense  (DoD)  is 
necessary  to  provide  balance.  In  doing  so,  it  is  readily  apparent  that  both  DoD  and  the 
private  sector  rely  almost  exclusively  on  commercial-off-the-shelf  (COTS)  acquisition 
for  the  introduction  of  new  information  technologies  and  upgrades  to  existing  systems. 

The  subject  of  widespread  COTS  integration  has  been  addressed  as  a  matter  of 
concern  by  NATO  in  a  recent  study. One  presenter  characterized  the  driving 
mechanisms  behind  COTS  acquisition  thus;’^"^ 

COTS  elements  are  produced  for  the  civil  mtirketplace,  and  evolve  in  the  context 
of  a  fine  balance  across  commercial  issues  of; 

•  cost  competiveness 

•  customer  expectation  of  quality 


NATO,  Visualisation  Study,  133. 

NATO,  COTS  Study. 

Ian  White,  “Wrapping  the  COTS  Dilemma”,  in  NATO,  COTS  Study,  1-1. 
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•  customer  tolerance  to  shortfalls  in  quality 

•  lifetime  in  the  marketplace 

•  commercial  through -life  support  needs 

•  time  to  market 

•  mechanisms  for  maximizing  a  future  market  share 

This  marketplace-driven  design  methodology  is  predicated  upon  generic  design 
that  provides  a  one-size-fits  all  package  that  maximizes  sales,  yet  fails  to  address  specific 
user  functionality  requirements. 

A  second  NATO  presenter  pointed  out  the  tightly-linked  relationship  between  the 
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private  and  military  sectors: 

The  ‘ruthless  pursuit  of  COTS’  is  increasing  the  penetration  of  unmodified  COTS 
technology  and  standards  in  the  military  domain.  Therefore,  as  the  defence 
community  becomes  more  reliant  on  off  the  shelf  products  and  standards,  it  is 
increasingly  a  stakeholder  in  the  results  of  the  civil  process. 

Contrasting  views  that  cast  the  COTS  concept  in  a  more  favorable  light  were  also 

expressed.  After  providing  several  examples  of  successful  COTS  integration,  one  U.S. 

presenter  offered  a  pragmatic  perspective:  “the  reality  is  that  the  significant  investments 

made  by  the  commercial  sector  in  information  technologies  are  orders  of  magnitude 

greater  than  the  U.S.  military  can  afford  to  drive  or  influence.”  Seemingly  expressed 

in  a  tone  of  resignation,  this  view  seems  to  submit  to  the  notion  that  the  commercial 

information  technologies  sector  operates  as  an  autonomous  juggernaut  churning  out  “IT 

solutions”  without  regard  to  customer  design  requirements. 

J.P.  Thorlby,  “The  Coordinated  Defence  Role  in  Civil  (Telecom)  Standardisation,”  NATO, 
COTS  Study,  6- 1 . 

James  J.  Barbarello  and  Walter  Kasian,  “United  States  Army  Commercial  Off-the-Shelf 
(COTS)  Experience:  The  Promises  and  Realities,”  NATO,  COTS  Study,  5-9. 
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While  perhaps  an  overly  stark  characterization,  the  COTS  phenomenon  fosters  a 
“delegated-design  mentality”  that  surrenders  design  initiative  to  the  vendor,  encouraging 
design-by-proxy.  Under  such  conditions,  the  critical  requirement  for  automated  tools 
designed  to  support  intelligence  analysis,  and  incorporating  the  requisite  cognitive, 
graphic  design,  and  human-computer-interface  factors,  would  occur  only  by  fortuitous 
accident,  if  at  all. 

The  foregoing  discussion  sheds  light  on  the  fact  that  COTS  acquisition,  even  if 
inexorable  and  perhaps  inevitable,  continues  to  be  a  source  of  concern.  If  we  are  to 
accept  COTS  procurement,  particularly  in  information  technologies,  as  the  current 
dominant  paradigm,  then  a  closer  examination  of  specific  DoD  guidance  regarding 
its  integration  is  in  order.  In  this  discussion,  the  human-computer  interface  (HCI)  is  of 
particular  concern.  A  brief  review  of  two  key  DoD  documents  that  specifically  address 
HCI  implementation  guidance  is  revealing. 

1  97 

The  Department  of  Defense  Intelligence  Information  System  (DoDIIS)  Profile 
is  designed  to  “provide  commonality  and  consistency  among  DoDIIS  mission  application 
development,  integration,  and  site  configuration  activities.”'^®  Not  intended  to  duplicate 
the  Joint  Technical  Architecture  (JTA)  document  (reviewed  below),  its  intent  is  to 
“ensure  systems  in  the  DoDIIS  are  closely  integrated  and  interoperable.”'^^ 

Further,  the  profile  “lists  the  JTA  standards  with  which  Government  Off-the-Shelf 
(GOTS)  software  must  be  compliant  [along  with]  Commercial  Off-the-Shelf  COTS 

™  Department  of  Defense,  Department  of  Defense  Intelligence  Information  System  (DoDIIS) 
Profile.  Version  4.1  (Washington,  DC:GPO,  November  2002).  Cited  hereafter  as  DoDIIS  Profile. 

DoDIIS  Profile,  ii. 

DoDIIS  Profile,  ii. 
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products  that  comply  with  the  appropriate  JTA  standards. Section  2  addresses 

service  areas,  “high-level  groupings  of  standards  that  comprise  a  common  set  of 

technical  capabilities,  such  as  User  Interface  standards.”’^'  Human  Computer 

1^2 

Interface  Standards  (HCI)  are  found  in  Section  2.4. 

A  review  of  Section  2.4  reveals  no  entry  for  “DoDIIS  Community-Specific 
Guidance”  regarding  GUI  (graphical  user  interface)  guides.  Under  “Mandated 
Standards,”  only  the  Microsoft  publication,  “Windows  Interface  Guidelines  for  Software 
Design”  is  offered  -  further  evidence  of  the  “delegated  design  principle”  in  action. 

The  Joint  Technical  Architecture  (JTA)  document  “defines  the  service  areas, 
interfaces,  and  standards  applicable  to  all  DoD  systems,  and  its  adoption  is  mandated  for 
the  management,  development,  and  acquisition  of  new  or  improved  systems  throughout 
the  DoD,”*^^  Divided  into  two  sections,  the  JTA  addresses  JTA  Core  and  JTA  domain 
issues,  including  the  domain  for  Command,  Control,  Communications,  Computers, 
Intelligence,  Surveillance,  and  Reconnaissance  (C4ISR)  where  intelligence-related 
system  standards  are  specified. 

Specific  excerpts  from  the  Joint  Technical  Architecture  document  indicate  a  noted 
absence  of  interface  design  guidance,  as  follows 

C4ISR.2.5  Human-Computer  Interface  Standards 
C4ISR.2.5.I  Introduction 

DoDIIS  Profile,  ii. 

DoDIIS  Profile,  9. 

DoDIIS  Profile,  33. 

Department  of  Defense,  Joint  Technical  Architecture  (JTA),  Version  4.0  (Washington,  DC: 
GPO,  April  2001),  iii.  Cited  hereafter  as  DoD,  Joint  Technical  Architecture. 

DoD,  Joint  Technical  Architecture,  96. 
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The  human-computer  interface  standards  and  profiles  described  in  this  section 

facilitate  interoperability  between  C4ISR  systems  through  the  use  of  standardized 

user  interfaces,  style  guides,  and  symbology. 

C4ISR.2.5.2  Mandated  Standards 

There  are  currently  no  mandated  standards  identified  in  this  service  area  of  the 

C4ISR  domain. 

C4ISR.2.5.3  Emerging  Standards 

There  are  currently  no  emerging  standards  identified  in  this  service  area  of  the 

C4ISR  domain. 

Cognitive  Task  Analysis  (CTA)  Methodologies 

The  previous  discussions  have  sought  to  emphasize  the  importance  of  considering 
“cognitive  process”  as  a  fundamental  design  element  in  systems  that  operate  within  the 
domain  of  intelligence  analysis.  A  wide  range  of  assessment  methodologies  are  available 
to  the  researcher  or  developer  that  can  assist  in  achieving  a  better  understanding  of  the 
cognitive  aspects  of  a  domain  or  a  group  of  users.  A  very  brief  introduction  to  Cognitive 
Task  Analysis  (CTA)  methodologies  is  provided  to  illustrate  one  such  approach. 

While  many  variations  of  the  CTA  concept  can  be  employed,  a  commonality  is 
generally  found  in  attempting  “to  provide  a  formal  specification  of  the  knowledge  and 
cognitive  processing  requirements  for  competent  performance  of  domain  tasks.”*^^ 

This  can  be  achieved  by  concentrating  on  analysis  of  the  domain  itself,  empirical 
approaches  to  understanding  the  tasks  performed,  or  computer-based  modeling.  In 
practice,  the  three  “idealized”  approaches  “should  be  viewed  as  complementary  rather 
than  as  alternative  methods  [as]  a  mix  of  analytical  and  empirical  techniques  are  required 
for  a  thorough  cognitive  task  analysis.”'^® 

Roth  and  others.  Cognitive  Engineering,  15. 

Roth  and  others.  Cognitive  Engineering,  16. 
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Numerous  studies,  many  of  which  are  cited  throughout  this  thesis,  clearly 
conclude  that  the  choice  of  metaphor  is  a  critical  component  in  human-computer  interface 
design  and  plays  an  active  role  in  shaping  the  representation  of  knowledge.  It  has  been 
pointed  out  that  the  choice  of  an  inappropriate  metaphor  can  “cripple  the  interface  with 
irrelevant  limitations  and  blind  the  designer  to  new  [and  more  appropriate]  paradigms.”’^® 
Further,  the  research  suggests  that  domain  relevance  is  lacking  in  business-model 
tool  sets  when  applied  to  the  problems  of  intelligence  analysis. 

The  negative  effects  of  the  PC  Paradigm  have  evolved  from  a  COTS-driven 
product-oriented  approach  and  represents  continued  extensions  of  the  desktop  metaphor 
to  domains  well  outside  its  original  design  intent.  The  current  paradigm  is  not  derived 
from  cognitive  ergonomic  requirements  unique  to  intelligence  analysis,  nor  does  it 
support  the  analytical  methodologies  widely  regarded  as  “best  practices”  within  the  field. 

These  conclusions  are  based  in  part  on  the  following  points  that  were  illuminated 
during  the  multidisciplinary  assessment. 

1.  The  author  contends  that  this  near-exclusive  reliance  on  COTS  technologies  is 
based  on  market-driven,  commercial  product  solutions  rather  than  a  “domain  specific” 
system  design  that  is  optimized  to  support  intelligence  analysis.  The  COTS  acquisition 
model  is  a  design-by-proxy  approach  that  has  surrendered  design  initiative  to  the  vendor. 
Domain  independence  is  manifested  as  a  generic,  one-size-fits-all  design  approach 
optimized  to  reach  the  widest  possible  audience.  While  commercially  impressive,  such 
an  approach  fails  to  address  the  needs  of  those  who  require  a  domain-specific  design, 

2.  To  ensure  suitability,  a  more  effective  approach  is  indicated  by  basing 
acquisition  and  development  decisions  on  design  principles  that  take  basic  cognitive 

Gentner  and  Nielson,  71. 
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A  comprehensive  articulation  of  the  cognitive  demands  involved  in  intelligence 
analysis  will  be  required  for  establishing  an  effective  design  specification  that  addresses 
the  concerns  of  domain  specificity  and  information  display  congruency  as  outlined  in  this 
thesis.  Cognitive  Task  Analysis,  in  one  or  all  of  its  many  possible  implementations,  is 

1  "^7 

offered  as  a  promising  approach. 


A  particularly  useful  reference  for  further  information  concerning  Cognitive  Task  Analysis  can 
be  found  online  at  CTA  Resource,  located  at  URL;<http://www.ctaresource.coni/>. 
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CHAPTER  3 


METHODOLOGY:  THE  GROUNDED  THEORY  MODEL 

A  grounded  theory  approach,  based  on  Leedy’s  model’ was  followed  in  the 
structure  of  this  study.  This  methodology  is  particularly  well  suited  “when  all  of  the 
concepts  pertaining  to  a  given  phenomenon  have  not  yet  been  identified,  at  least  not 
in  this  population  or  place;  or  [when]  the  relationships  between  the  concepts  are  poorly 
understood  or  conceptually  underdeveloped.”’^^  Such  an  approach  was  selected  to 
illustrate  the  conceptual  density  of  this  multidisciplinary  study  and  explore  important 
linkages  through  the  use  of  theoretical  sampling}'^ 

Leedy’s  depiction  of  the  suitability  of  a  grounded  theory  approach,  provided 
above,  is  especially  apt  for  the  purposes  of  this  thesis.  In  examining  the  research 
question,  it  quickly  became  evident  that  many  of  the  relevant  concepts  and  their 
relationships  appear  to  fit  the  description  of  being  poorly  understood  or  insufficiently 
developed  within  the  “population  or  place”  of  the  intelligence  domain.  This  has  had  the 
effect  of  creating  “conceptual  islands”  of  theory  that  often  exist  in  relative  isolation 
within  their  respective  fields  and  has  lacked  a  coordinated  synthesis  as  applied  to 
intelligence  problems. 


Paul  D.  Leedy,  Practical  Research:  Planning  and  Design,  6'*'  ed.  (Upper  Saddle  River,  NJ; 
Merrill  Prentice  Hall,  1997);  162-165. 

Leedy,  163. 

'""Leedy,  164. 
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Rather  than  relying  upon  a  limited  or  narrowly-defined  body  of  evidence,  a 
“broad  and  deep”  approach  was  utilized.  Identifying  commonality  in  related,  but 
disparate,  theoretical  and  applied  techniques  thus  allowed  a  convergence  of  interlocking 
themes  to  emerge. 

The  research  sought  to  capture  and  distill  the  relevant  findings  of  diverse  sources 
and  perspectives  comprising  case  studies,  research  proposals,  interview  compilations, 
journal  articles,  published  and  unpublished  papers,  related  theses,  proposed  and  fielded 
models,  and  product  literature.  Balance  was  enhanced  by  including  the  views  derived 
from  academia,  private  industry,  commercial  sources,  and  Department  of  Defense 
research  studies.  Both  U.S.  and  foreign  perspectives  were  considered  and  evaluated. 

Special  emphasis  was  placed  on  DoD-related  studies  that  explored  cognitive 
aspects  relating  to  the  intelligence  domain  such  as  the  primary  research  conducted  by 
Patterson,  Roth  and  Woods*'*^',  and  NATO-sponsored  research  in  visualization  of 
massive  military  datasets. 

Chapter  2  documents  the  wide-ranging  survey  that  was  required  to  identify  and 
“pull  together”  the  disparate  “islands”  of  theory  and  practice  that  are  openly  available,  yet 
often  “buried”  in  the  literature  of  their  respective  fields  of  interest.  The  author’s 
contention  is  that  a  unified  or  coordinated  approach  to  incorporating  such  design  factors 
into  our  intelligence  systems  is  non-existent,  undermining  the  analytical  effort. 

The  research  methodology  was  focused  on  presenting  a  logical  progression  that 
was  designed  to  flow  in  the  following  manner; 

The  study  cited  as  “Roth  and  others.  Cognitive  Engineering,’’’  and  Aiding  the  Intelligence 
Analyst  in  Situations  of  Data  Overload:  A  Simulation  Study  of  Computer-Supported  Inferential  Analysis 
Under  Data  Overload  by  Patterson,  Roth,  and  Woods  (cited  previously)  were  particularly  useful. 

NATO,  Visualisation  Study, 
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a.  First,  examine  and  characterize  the  current  tool  available. 

(Existing  paradigm,  COTS  influence,  desktop  metaphor) 

b.  Second,  examine  the  domain  of  interest  in  which  the  tool  is  applied. 
(Specifically,  the  domain  of  intelligence  analysis). 

c.  Third,  examine  the  relevant  fields  of  study  that  directly  influence  the 
tool  and  point  out  critical  linkages  where  they  occur. 

(Cognitive  science,  graphics  design,  human-computer  interface) 

d.  Fourth,  examine  the  incorporation  of  such  principles  into  existing 
design  guidance. 

(Published  DoD  guidance  and  NATO  perspectives) 

e.  The  final  category,  Cognitive  Task  Analysis  Methodologies,  concludes 
the  discussion  of  concepts  by  introducing  a  potential  solution  to 
address  shortcomings  and  design  concerns  such  as  those  raised  in  the 
thesis. 

In  conducting  the  research,  certain  evaluative  criteria  were  used  to  ascertain 
whether  a  deleterious  effect  on  analysis  could  be  anticipated  or  expected.  By  examining 
the  theoretical  and  conceptual  principles  that  guide  generally-accepted  design,  such  an 
assessment  could  identify  where  discontinuity  or  design  conflict  may  occur.  Within  this 
thesis,  the  following  criteria  were  used  to  indicate  the  potential  for  the  current  personal 
computer  environment  to  function  as  an  analysis  inhibitor; 

•  Dissatisfaction  with  the  current  paradigm. 

•  Evidence  to  suggest  that  analyst  tasks  may  be  inadequately  supported. 

•  Evidence  to  suggest  that  the  current  paradigm  appears  to  violate  or  ignore 
accepted  guidelines  within  cognitive  science,  human-computer  interface 
(HCI)  and  graphic  design. 

•  Evidence  that  congruency  issues  may  be  apparent. 
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The  complexities  of  handling  a  multidisciplinary  review  was  managed  by 
arranging  sources  into  the  following  ten  subjects  or  “fields  of  study”  categories  that 
facilitated  the  evaluation  of  applicability  and  relevance. 

•  A  Discussion  of  Paradigm,  Theory,  and  Scientific  Endeavor 

•  A  Look  at  the  Current  Model 

•  Intelligence  Analysis  as  Cognitive  Process 

•  Applicable  Concepts  of  Psychology  and  Cognitive  Science 

•  Visualization  and  Mental  Models 

•  Tenets  of  Graphic  Design 

•  Human-Computer  Interface  (HCI)  Design  Considerations 

•  European/NATO  Perspective 

•  Department  of  Defense  (DoD)  Guidance 

•  Cognitive  Task  Analysis  (CTA)  Methodologies 
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CHAPTER  4 


FINDINGS 

As  a  qualitative  study,  the  research  explored  a  range  of  related  fields  and  sub¬ 
disciplines,  identifying  conceptual,  theoretical,  and  practical  linkages  that  appertain  to  the 
research  question.  To  assist  in  demonstrating  the  relevance  of  apparent  linkages  and 
where  they  seemed  to  occur,  the  research  question  is  re-stated  as  follows: 

Could  the  current  personal  computer  environment  (characterized  as  the  PC 
Paradigm)  function  as  an  analysis  inhibitor,  rather  than  as  a  tool  to  facilitate  analysis? 

By  examining  this  question  in  the  context  of  the  various  “fields  of  study” 
categories  outlined  in  Chapter  3  (Methodology),  the  research  illuminated  areas  where  the 
PC  Paradigm  environment,  even  theoretically,  may  indeed  foster  conditions  detrimental 
to  analysis.  Key  findings  of  this  assessment  are  presented  below.  Discussion  and 
conclusions  drawn  from  such  observations  are  reserved  for  the  following  chapter. 

Key  Findings 

Congruency  or  “Fitness-of-Purnose”  Considerations.  The  increasing 
sophistication  of  intelligence  analysis  methodologies  requires  tools  that  support 
high-level  cognitive  processes  such  as  inferential  reasoning  and  visualization  of  complex 
or  “massive”  data.  The  existing  PC-based  systems  utilize  a  business-model  created  for  a 
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general-purpose  commercial  market,  employing  a  desktop  metaphor  (office-based)  user- 
interface.  A  fundamental  paradigm  conflict  is  suggested  between  the  requirements  of  the 
analytic  process  and  the  limitations  imposed  by  the  desktop  metaphor. 

The  Presence  of  Conceptual  Islands.  Significant  and  relevant  research  has 
been  conducted,  and  is  ongoing,  in  many  fields  that  lie  outside  the  normal  purview  of  the 
intelligence  community.  The  literature  is  replete  with  studies  that  remain  isolated  in 
specialized  domains  although  cross-field  application  is  evident.  Such  observations  point 
to  a  review  of  the  design  approaches,  as  suggested  below. 

Interdisciplinary  Design  Approaches  Required.  Cognitive  studies  cited  in  the 
thesis  suggest  that  the  process  of  intelligence  analysis  is  considerably  influenced  by 
information  representation,  mental  modeling,  and  visualization.  Even  though  this 
appears  to  be  a  fundamental  understanding,  a  systematic  or  coordinated  effort  to 
synthesize  these  generally-accepted  concepts  into  application  design  guidance  appears  to 
be  lacking. 

COTS  Influence.  The  role  of  Commercial-Off -The-Shelf  (COTS)  technology 
acquisition  is  tightly  linked  to  the  discussion  of  tool  suitability  and  its  influence  cannot  be 
discounted.  Attempts  to  assess  whether  the  current  paradigm  inhibits  or  facilitates  the 
intended  purpose  of  intelligence  analysis  return  in  circular  fashion  to  the  origin  of  the  tool 
and  the  design  criteria  (or  the  absence  of  design  criteria)  that  informed  its  development. 
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Proactive  NATO  Efforts.  NATO  research*'*^  indicates  a  solid  appreciation  for 
incorporating  cognitive  principles  into  defense-related  information  display  design  and 
have  adopted  cognitive  reference  models.  Additionally,  NATO  studies  have  expressed 
concern  on  the  pervasive  and  potentially  negative  aspects  of  continued  COTS  technology 
acquisition.*"*^  The  level  of  coordination  between  NATO  and  U.S.  design  initiatives 
was  not  accessed. 

Minimal  DoD  Guidance.  In  two  key  technical  architecture  documents,’"*^ 
little  or  no  design  guidance  is  offered  in  regards  to  human-computer  interface  issues. 
Rather  than  domain-specific  concerns  (i.e.  intelligence  analysis)  driving  interface 
requirements,  commercial  standards  are  adopted. 

A  Tangential  Finding:  Issues  of  Ontology*"*^ 

This  thesis  did  not  undertake  an  examination  of  ontological  issues  or  related 
topics  of  taxonomy  and  lexicon  development.  It  became  apparent  during  the  research 
however,  that  the  choice  of  appropriate  metaphor  candidates  and  consequent  interface 
design  relies  heavily  on  a  common  understanding  between  the  user  and  the  developer  of 
domain  concepts  and  conceptual  relationships  -  both  issues  of  ontology. 

A  key  detractor  to  a  better  understanding  of  the  cognitive  processes  involved 
within  intelligence  analysis  is  the  lack  of  an  accepted  ontological  schema  within  the  IC 

NATO,  Visualisation  Study. 

NATO,  COTS  Study. 

DoDIlS  Profile  and  DoD,  Joint  Technical  Architecture. 

Sowa,  294.  Sowa  defines  an  ontology  as  resulting  from  a  conceptual  analysis  whereby  a 
“precise,  formalizable  catalog  of  concepts,  relations,  facts,  and  principles”  is  created. 
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itself.  Such  an  ontology  is  essential  in  defining  terms  and  conceptual  relationships  that 
provide  the  necessary  “lattice- work”  to  facilitate  cognitive  mapping  of  the  domain. 

The  lack  of  an  accepted  ontology  complicates  development  of  standardized  icon-sets  and 
the  creation  of  shared  metaphors.  Both  are  fundamental  tasks  in  designing  a  user- 
interface  in  any  domain.  The  domain  of  intelligence  analysis  is  no  exception.'**^ 


Promising  research  in  ontological  design,  with  specific  application  to  the  intelligence 
community,  is  ongoing.  A  partial  list  of  involved  organizations  include  the  Defense  Advanced  Research 
Projects  Agency  (DARPA),  the  Advanced  Research  and  Development  Activity  (ARDA),  and  the  Advanced 
Analysis  Lab  at  NS  A. 
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CHAPTER  5 


SUMMARY  /CONCLUSIONS  /  RECOMMENDATIONS 

SUMMARY 

The  intelligence  cycle,  whether  depicted  in  classic  circular  flow  or  as  an  iterative, 

looping  model,  depends  on  the  analytical  process  at  its  core.  Even  with  the  widespread 

use  of  automated  systems,  this  task  currently  remains  a  mental  process,  performed  by  the 

intelligence  analyst.  While  such  systems  greatly  facilitate  other  aspects  of  the 

intelligence  cycle,  particularly  in  collection  and  dissemination,  analysis  relies  on  an 

interplay  of  cognitive  factors  that  have  yet  to  be  replicated  in  fielded  computer-based 

tools.  The  complexity  of  the  problem  is  described  in  a  recent  study. 

Sometimes  developers  believe  that  shifting  the  task  to  a  computer  somehow 
makes  the  cognitive  challenges  of  focusing  in  on  the  relevant  subset  disappear.  In 
fact,  all  finite  cognitive  processors  face  this  challenge,  whether  they  are  an 
individual,  a  machine  agent,  a  human-machine  ensemble,  or  a  team  of  people.  It 
always  takes  cognitive  work  to  find  the  significance  in  data. 

In  an  effort  to  augment  the  analyst’s  workload  with  current  technological  tools, 

the  Department  of  Defense  has  adopted  the  industry- standard  personal  computer  as  a 

common  solution.  This  has  been  achieved  by  relying  on  Commercial-Off-The-Shelf 

(COTS)  acquisition  as  the  primary  means  to  introduce  and  upgrade  computer 

technologies  throughout  the  military. 


Patterson  and  others.  Aiding  the  Intelligence  Analyst,  7 , 
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In  adopting  this  approach,  the  personal  computer,  or  “PC,”  has  become  a  de-facto 
standard,  along  with  its  commercial,  business-model  design.  Within  the  thesis,  this 
common,  and  widely-accepted  desktop  computer  environment  is  defined  as  the  PC 
Paradigm.  The  definition  encompasses  the  PC’s  typical  office-suite  software,  the 
information  display,  and  the  use  of  the  desktop  metaphor  as  the  user-interface. 

The  research  question  inspired  by  the  apparent  dominance  of  the  PC  Paradigm 
was  stated  as:  “Could  the  personal  computer  environment  (characterized  as  the  PC 
Paradigm)  function  as  an  analysis  inhibitor,  rather  than  a  tool  to  facilitate  analysis?” 

This  question  was  assessed  by  conducting  a  multidisciplinary  exploration  of 
related  fields  of  study,  including  the  areas  of  cognitive  science,  graphic  design,  and 
human-computer  interface  (HCI)  considerations,  along  with  a  review  of  NATO  and  DoD 
design  guidance  on  the  subject  of  HCI  design.  This  expansive  review  revealed 
significant  intersections  in  design  philosophy,  applicable  models,  and  concepts  that 
indicated  that  the  cognitive  aspects  of  information  displays  are  critical  design 
considerations  in  determining  how  information  is  interpreted  by  the  user.  The  concept  of 
congruency  or  conformity  was  established  as  an  important  criteria  in  determining 
whether  an  information  display  maintained  a  “fitness-of-purpose.”  Simply  stated,  does 
the  tool  facilitate  or  inhibit  the  work  it  is  designed  to  assist? 

Key  findings  suggested  that  the  reliance  on  COTS  equipment  has  significant 
implications  that  are  tightly  linked  to  the  discussions  of  the  personal  computer  as  a  tool 
for  intelligence  analysis.  Specifically,  the  commercial  design  of  the  PC  facilitates  its 
mass-market  distribution  but  does  not  address  the  domain-specific  requirements  of  the 
intelligence  community. 
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Several  research  studies,  including  NATO  research  and  DoD-sponsored  projects, 
underscored  the  critical  importance  of  considering  cognitive  design  principles  in  the 
development  of  information  display  and  visualization  tools.  A  lack  of  DoD  guidance  in 
such  matters  was  apparent  however,  in  the  technical  architecture  documents  reviewed. 

CONCLUSIONS 

Intelligence  analysis,  as  a  profession,  is  inherently  interdisciplinary  and  often 
draws  upon  knowledge  and  skills  from  multiple  knowledge  domains.  Such  cross¬ 
fertilization  of  ideas  is  to  be  encouraged  and  should  be  expanded  upon  to  avoid  the 
tendency  to  limit  solution  possibilities  to  those  within  immediate  grasp.  This  research 
has  sought  to  offer  a  linkage  to  ideas  and  concepts  that  are  usually  found  outside  the 
normal  scope  of  the  intelligence  analyst.  Potential  answers  to  the  research  question  were 
believed  to  be  found  at  the  intersection  of  several  fields  of  study  (or  domains),  each 
sharing  an  overlapping  influence  on  some  aspect  of  the  question. 

The  results  of  the  thesis  research  indicate  that  the  potential  for  inhibited  analysis 
exists  within  the  currently  accepted  PC  Paradigm.  It  is  the  author’s  contention  that  the 
industry-standard  PC  environment,  and  its  desktop  metaphor,  are  ill-suited  for  the  task  of 
intelligence  analysis.  A  fundamental  difference  exists  between  data  representation  and 
analysis,  with  the  former  merely  retrieving  and  arraying  the  data,  while  the  latter 
represents  human  cognitive  processes.  The  current  tools  of  the  PC  Paradigm  support  the 
tasks  of  retrieval,  representation,  and  dissemination,  while  the  process  of  analysis  remains 
largely  unaided. 
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engineering  considerations  into  account.  The  following  questions  are  examples  of  the 
type  of  elicitation  required  to  define  the  domain  specificity  required: 

•  What  are  the  goals  and  constraints  of  the  intelligence  analysis  domain? 

•  What  range  of  tasks  do  the  analysts  perform? 

•  What  analytical  methodologies  do  they  use? 

•  What  factors  contribute  to  analytical  complexity? 

3.  The  emphasis  should  be  placed  on  process  and  goal  and  not  product  or  tool. 
The  desired  emphasis  is  on  the  goal  of  information  transference  and  subsequent 
knowledge  formation  and  not  on  the  technology  involved 

4.  Congruency  should  serve  as  a  basic  criteria  to  assess  whether  a  display 
facilitates  or  inhibits  the  task  it  is  claimed  to  support.  There  is  scant  evidence  to  suggest 
that  such  criteria  are  considered  as  design  factors  or  evaluative  measures  in  the  current 
paradigm.  The  concept  of  visualization  as  a  cognitive  process  seems  to  be  broadly 
understood  in  the  design  community,  but  such  an  appreciation  seems  less  evident  in  the 
tools  currently  available  to  support  intelligence  analysis. 

5.  The  crucial  role  of  cognitive  process  as  a  fundamental  component  of 
intelligence  analysis  has  been  recognized  for  over  two  decades.  Previous  studies, 
rather  than  comprising  elements  of  a  core  literature  that  informs  system  design  for 
automated  tools  to  support  intelligence  analysis,  remain  obscure  and  are  not  integrated 
into  a  coherent  design  methodology 

6.  NATO  research  endorses  proactive  design  rather  than  passive  acceptance  of 
COTS  technologies  and  has  accepted  cognitive  reference  models  to  inform  the 
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development  process,  In  contrast,  DoD  guidance  is  minimal  and  specifically  states 
within  the  Joint  Technical  Architecture  document  that; 

‘There  are  currently  no  mandated  (or  emerging)  standards  identified  [in  human- 
computer  interface  standards  within]  the  C4ISR  domain.”^^'’ 

It  is  significant  to  note  with  particular  emphasis  the  specific  phraseology  of  this 
excerpted  passage,  “currently  no  mandated  or  emerging  standards.” 

1.  Other  domains  (including  NASA  examples  and  others)  have  rejected  or 
expressed  dissatisfaction  with  the  desktop  metaphor.  Numerous  examples  exist  in  the 
literature  that  argue  for  a  replacement  of  the  dominant  office-based  desktop  metaphor  on 
the  basis  of  its  unsuitability  and  lack  of  domain  specificity  in  their  respective  fields  of 
endeavor. 


RECOMMENDATIONS  AND  OPPORTUNITIES  FOR  FUTURE  STUDY 

Configuration  Management  for  Cognitive  Systems 

A  centralized  means  of  ensuring  “cognitive  systems  configuration  management” 
is  recommended  to  facilitate  design  oversight.  Presently,  promising  research  and 
development  activity  is  undertaken  largely  in  isolation  resulting  in  a  fragmented, 
uncoordinated  effort  that  fails  to  identify  and  capture  “best  practices”  in  systems  design. 


DoD  Joint  Technical  Architecture,  96. 
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Further  Exploration  of  CTA  Methods 

Cognitive  Task  Analysis  methods  were  reviewed  as  a  promising  tool  to  assist  in 
modeling  the  cognitive  environment  within  the  intelligence  domain.  Such  structured 
methodologies  could  potentially  provide  the  “verifiable  and  replicable”  data  that  is 
missing  within  many  qualitative  assessments.  Cognitive  Task  Analysis  (CTA)  is  an 
umbrella  term  that  encompasses  a  wide  range  of  potential  tools  and  evaluative  concepts 
that  has  only  been  introduced  here  in  minimal  depth.  Further  research  that  incorporates 
CTA  methodologies  or  evaluates  their  utility  in  addressing  IC  issues  is  warranted  for 
future  studies. 

Increased  Ontology  Development  Within  The  IC 

The  issue  of  ontology  and  the  requirement  for  its  articulation  as  a  matter  of 
interface  design  was  mentioned  in  Chapter  4  as  a  “tangential  finding.”  In  the  author’s 
opinion,  it  is  likely  that  bringing  the  entire  “intelligence  community”  in  toto  under  a 
single  ontological  umbrella  is  overly  ambitious.  A  more  pragmatic  goal  would  perhaps 
be  achieving  a  consensus  or  “baseline  agreement”  as  a  starting  point  within  the  individual 
sub-disciplines  such  as  IMINT,  SIGINT,  HUMINT,  MASINT.  Until  a  minimal  level  of 
“lexical  consistency”  is  reached,  efforts  to  optimize  a  comprehensive  design  will  likely 
remain  fragmented,  unfocused,  and  counterproductive. 
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Visualization  Implications  for  Imagery  Analysis 

Fischbein’s  observation  that  “  a  visual  image  not  only  organizes  the  data  at  hand 
in  meaningful  structures,  but  is  also  an  important  factor  guiding  the  analytical 
development  of  a  solution”^^^  is  particularly  revealing. 

In  the  case  of  imagery  analysis,  the  “data  at  hand”  are  already  present  in  visual 
form  -  it  is  visual  data.  For  electro-optical  (EO)  imagery,  the  data  closely  approximates  a 
literal  model,  a  visual  wavelength  representation  of  the  target  as  it  would  appear 
monochromatic  ally.  Other  sensors  (IR,  radar,  MSI)  represent  non-visible  wavelength 
depictions  and  are  therefore  figurative  or  non-literal  models  of  the  scene  constructed  of 
the  spectral  building  blocks  inherent  to  that  sensor.  In  a  very  real  sense,  imagery 
products  of  all  types  are  therefore  visualizations  in  their  own  right,  consisting  of  families 
of  literal  and  non-literal  models.  This  would  suggest  that  any  automated  system 
approach  to  augment  imagery  analysis  may  involve  “visualizations  o/ visualizations”  at 
the  fundamental  or  cognitive  level.  Additional  research  to  further  investigate  this 
contention  may  prove  rewarding. 


Towards  A  New  Vision 

A  final  comment  in  regards  to  future  direction  is  offered.  Wexelblat  has  pointed 
out  that  an  article  originally  appearing  in  Byte  magazine  in  1992  suggested  that 
“research  from  many  fields  is  being  synthesized  to  create  a  design  philosophy  of 
information  environments.”  While  an  intriguing  possibility,  the  literature  offers  little 


Fischbein,  104. 

Charles  Grantham,  “Visualization  of  Information  Flows:  Virtual  Reality  as  an  Organizational 
Modeling  Technique,”  in  Virtual  Reality:  Applications  and  Explorations,  ed.  Alan  Wexelblat  (Cambridge, 
MA:  Academic  Press  Professional,  1993):  220. 
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evidence  that  such  a  synthesis  has  yet  occurred.  Instead,  research  efforts  often  remain 
fragmented,  and  can  often  be  characterized  as  in  the  following  excerpt  from  a  recent 
NATO  study: 

We  have  accumulated  a  lot  of  lessons  learned,  rules  of  thumb,  best  practices,  and  ways  to 
avoid  pits  into  which  we  have  already  fallen.  However,  the  half-life  of  these  lessons  from 
the  past  will  continue  to  decline  at  the  same  (accelerating)  pace  as  that  of  computer 
technology.  The  only  solution  will  be  to  design  [automated  support  systems]  based  on  a 
coherent,  human-centered  philosophy. 

The  renewed  pursuit  of  the  coordinated  “design  philosophy  of  information 
environments”  remains  a  worthy,  and  elusive,  goal  and  should  receive  sustained  support. 

Cited  research  studies  indicate  that  both  the  EU  and  NATO  have  made  proactive 
steps  to  structure  design  initiatives  that  take  such  considerations  into  account  early  in  the 
design  process.  Within  the  U.S.,  The  Advanced  Research  and  Development  Activity 
(ARDA)  is  an  important  example  of  a  forward-looking  approach  that  emphasizes  analytic 
process  over  off-the-shelf  solutions.  A  broad  reevaluation  of  whether  a  COTS-centiic 
approach  is  fundamentally  incompatible  with  efforts  to  create  a  coherent  design 
philosophy  appears  to  be  warranted. 


Sandra  G.  Hart,  “Human  Factors  Directions  For  Civil  Aviation,”  in  North  Atlantic  Treaty 
Organization,  Human  Factors,  9-12  to  9-13. 
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